Monday, March 14, 2022

[SOLVED] Springboot server in Elastic Beanstalk creates files that I can't see

Issue

I have a Springboot server that is deployed to an Elastic Beanstalk environment in AWS. The basic functionality is this:

 1. Upload a file to the server
 2. The server processes file by doing some data manipulation.
 3. Then the file that is created is sent to a user via email.

The strange thing is that, the functionality mentioned above is working. The output file is sent to my email inbox successfully. However, the file cannot be seen when SSHed into the instance. The entire directory that gets created for the data manipulation is just not there. I have looked everywhere.

To test this, I even created a simple function in my Springboot Controller like this:

    @GetMapping("/")
    public ResponseEntity<String> dummyMethod() {
        // TODO : remove line below after testing
        new File(directoryToCreate).mkdirs();
        return new ResponseEntity<>("Successful health check. Status: 200 - OK", HttpStatus.OK);
    }

If I use Postman to hit this endpoint, the directory CANNOT be seen via the terminal that I am SSHed into. The program is working so I know that the code is correct in that sense, but its like the files and directories are invisible to me.

Furthermore, if I were to run the server locally (using Windows OR Linux) and hit this endpoint, the directory is successfully created.

Update: I found where the app lives in the environment at /var/app. But my folders and files are still not there, only the source code files, ect are there. The files that my server is supposed to be creating are still missing. I can even print out the absolute path to the file after creating it, but that file still doesn't exist. Here is an example:

            Files.copy(source, dest);
            logger.info("Successfully copied file to: {}", dest.getAbsolutePath());

will print...

Successfully copied file to: /tmp/TESTING/Test-Results/GVA_output_2021-12-13 12.32.58/results_map_GVA.csv

That path DOES NOT exist in my server, but I CAN send it to me via email from the server code after being processed. But if I SSH into the instance and go to that path, nothing is there.

If I use the command: find . -name "GVA*" (to search for the file I am looking for) then it prints this:

./var/lib/docker/overlay2/fbf04e23e39d61896a1c935748a63f2d3836487d9b166bae490764c30b8870ae/diff/tmp/TESTING/Test-Results/GVA_output_2021-12-09 18.15.59
./var/lib/docker/overlay2/fbf04e23e39d61896a1c935748a63f2d3836487d9b166bae490764c30b8870ae/diff/tmp/TESTING/Test-Results/GVA_output_2021-12-13 12.26.34
./var/lib/docker/overlay2/fbf04e23e39d61896a1c935748a63f2d3836487d9b166bae490764c30b8870ae/diff/tmp/TESTING/Test-Results/GVA_output_2021-12-13 12.32.58
./var/lib/docker/overlay2/fbf04e23e39d61896a1c935748a63f2d3836487d9b166bae490764c30b8870ae/merged/tmp/TESTING/Test-Results/GVA_output_2021-12-09 18.15.59
./var/lib/docker/overlay2/fbf04e23e39d61896a1c935748a63f2d3836487d9b166bae490764c30b8870ae/merged/tmp/TESTING/Test-Results/GVA_output_2021-12-13 12.26.34
./var/lib/docker/overlay2/fbf04e23e39d61896a1c935748a63f2d3836487d9b166bae490764c30b8870ae/merged/tmp/TESTING/Test-Results/GVA_output_2021-12-13 12.32.58

But this looks like it is keeping track of differences between versions of files since I see diff and merged in the file path. I just want to find where that file is actually residing.


Solution

If you need to store an uploaded file somewhere from a Spring BOOT app, look at using an Amazon S3 bucket as opposed to writing the file to a folder on the server. For example, assume you are working with a Photo app and the photos can be uploaded via the Spring BOOT app. Instead of placing this in a directory on the server, use the Amazon S3 Java API to store the file in an Amazon S3 bucket.

Here is an example of using a Spring BOOT app and handling uploaded files by placing them in a bucket.

Creating a dynamic web application that analyzes photos using the AWS SDK for Java

This example app also shows you how to use the SES API to send data (a report in this example) to a user via email.



Answered By - smac2020
Answer Checked By - Marilyn (WPSolving Volunteer)
at