Fork us on GitHub

Exif Orientation Tag and Smart Downloads

A couple of useful new APIs from Francesco Galgani
Exif Orientation Tag and Smart Downloads

Exif Orientation Tag and Smart Downloads

On some devices, Capture APIs return images with the correct orientation, meaning that they do not need to be changed to display correctly; on other devices, they return images with a fixed orientation and an EXIF tag that indicates how they must be rotated or flipped to display correctly.

More precisely, the Orientation Tag indicates the orientation of the camera with respect to the captured scene and can take a value from 0 to 8, as illustrated on the page Exif Orientation Tag. For testing purposes, you can download landscape and portrait images with all possible orientation values from the EXIF Orientation-flag example images repository.

What happens if we ignore the Orientation Tag

Suppose we acquire an image with the following code:

Form hi = new Form("Capture Test", BoxLayout.y());
Button button = new Button("Take Photo");
ScaleImageLabel photoLabel = new ScaleImageLabel();
hi.addAll(button, photoLabel);
hi.show();

button.addActionListener(l -> {
    String photoTempPath = Capture.capturePhoto();
    if (photoTempPath != null) { (1)
        try {
            String photoStoragePath = "myPhoto.jpg";
            Util.copy(FileSystemStorage.getInstance().openInputStream(photoTempPath), Storage.getInstance().createOutputStream(photoStoragePath)); (2)
            photoLabel.setIcon(EncodedImage.create(Storage.getInstance().createInputStream(photoStoragePath), Storage.getInstance().entrySize(photoStoragePath))); (3)
            hi.revalidate();
        } catch (IOException ex) {
            Log.p("Error after capturing photo", Log.ERROR);
            Log.e(ex);
            Log.sendLogAsync();
        }

    }
});

A few remarks:

1 photoTempPath is null if the user has cancelled the photo capture;
2 in this case, copying the file from the FileSystemStorage temporary folder to a "secure" location in FileSystemStorage or Storage is not strictly necessary, but it is a good habit that in certain circumstances prevents issues;
3 it is always preferable to use EncodedImage when we want to keep the impact on memory low.

Desired result

On my iPhone, the image is always in portrait orientation:

85854279 64b80800 b7b4 11ea 88e2 98a6d57fc09f

Unwanted result

This is the case of my Samsung Galaxy, the image was taken in portrait, but shown in a different orientation:

85853565 050d2d00 b7b3 11ea 8053 695772ecbfd0

Solving This Problem

All it takes is a small change to the code to solve this issue.

Just replace:

String photoStoragePath = "myPhoto.jpg";
Util.copy(FileSystemStorage.getInstance().openInputStream(photoTempPath), Storage.getInstance().createOutputStream(photoStoragePath));
photoLabel.setIcon(EncodedImage.create(Storage.getInstance().createInputStream(photoStoragePath), Storage.getInstance().entrySize(photoStoragePath)));

with:

String photoSafePath = FileSystemStorage.getInstance().getAppHomePath() + "/myPhoto.jpg"; (1)
Image img = Image.exifRotation(photoTempPath, photoSafePath, 1000); (2)
photoLabel.setIcon(img); (3)
1 In this case, we have to use FileSystemStorage rather than Storage due to a limitation of the exifRotation API, which maybe will be solved in the future;
2 the third parameter is optional, but as explained in the exifRotation Javadoc, the rotation of a high-resolution image is very inefficient, it is better to set the maximum size (width or height) that the image can assume, in this case 1000px, to obtain a significant advantage in processing time on less performing devices;
3 note that the instance of the Image object returned by exifRotation is an EncodedImage, to keep the impact on memory low.

Final result

On my iPhone the result is the same (as expected), while on my Android, taking the same photo, I get:

85862326 e82c2600 b7c1 11ea 9135 657f2a0eead7

This is the desired result. As a final note, I mention the Image.getExifOrientationTag API which allows you to get the EXIF orientation tag of an image, if available.

Network error resistant downloads with automatic resume

When we download a big file, such as a high-resolution image or a video, there are problems that can prevent the download from finishing:

  • The user moves the app to the background or external conditions (such as a phone call) move the app to the background;

  • The operating system enters power saving mode;

  • The Internet connection is lost or any other network error interrupts the download.

A server-side error may also occur, but this cannot be resolved client-side. All the other circumstances mentioned above can, provided that the server supports partial downloads via HTTP headers. Fortunately, this is a feature available by default on most common servers (such as Apache or Spring Boot). Almost all download managers allow to resume interrupted downloads, so I thought it was important to add such a feature to Codename One.

The solution

THe new API Util.downloadUrlSafely safely download the given URL to the Storage or to the FileSystemStorage.

This method is resistant to network errors and capable of resume the download as soon as network conditions allow and in a completely transparent way for the user.

Server requirements

The server must correctly return the Content-Length header and it must supports partial downloads.

Global network error handling requirements

In the global network error handling, there must be an automatic .retry() of the ConnectionRequest in the case of a network error.

I think the best way to show the use of this API is an actual complete example, which you can try as it is in the Simulator and on real devices:

public class MyApplication {

    private Form current;
    private Resources theme;

    public void init(Object context) {
        // use two network threads instead of one
        updateNetworkThreadCount(2);

        theme = UIManager.initFirstTheme("/theme");

        // Enable Toolbar on all Forms by default
        Toolbar.setGlobalToolbar(true);

        // Pro only feature
        Log.bindCrashProtection(true);

        // Manage both network errors (connectivity issues) and server errors (codes different from 2xx)
        addNetworkAndServerErrorListener();
    }

    public void start() {
        if(current != null){
            current.show();
            return;
        }

        String url = "https://www.informatica-libera.net/video/AVO_Cariati_Pasqua_2020.mp4"; // 38 MB

        Form form = new Form("Test Download 38MB", BoxLayout.y());
        Label infoLabel = new Label("Starting download...");
        form.add(infoLabel);

        try {
            Util.downloadUrlSafely(url, "myHeavyVideo.mp4", (percentage) -> {
                // percentage callback
                infoLabel.setText("Downloaded: " + percentage + "%");
                infoLabel.repaint();
            }, (filename) -> {
                // file saved callback
                infoLabel.setText("Downloaded completed");
                int fileSizeMB = Storage.getInstance().entrySize(filename) / 1048576;
                form.add("Checking files size: " + fileSizeMB + " MB");
                form.revalidate();
            });
        } catch (IOException ex) {
            Log.p("Error in downloading: " + url);
            Log.e(ex);
            form.add(new SpanLabel("Error in downloading:\n" + url));
            form.revalidate();
        }

        form.show();


    }

    public void stop() {
        current = getCurrentForm();
        if(current instanceof Dialog) {
            ((Dialog)current).dispose();
            current = getCurrentForm();
        }
    }

    public void destroy() {
    }

    private void addNetworkAndServerErrorListener() {
        // The following way to manage network errors is discussed here:
        // https://stackoverflow.com/questions/61993127/distinguish-between-server-side-errors-and-connection-problems
        addNetworkErrorListener(err -> {
            // prevents the event from propagating
            err.consume();

            if (err.getError() != null) {
                // this is the case of a network error,
                // like: java.io.IOException: Unreachable
                Log.p("Error connectiong to: " + err.getConnectionRequest().getUrl(), Log.ERROR);
                // maybe there are connectivity issues, let's try again
                ToastBar.showInfoMessage("Reconnect...");
                Timer timer = new Timer();
                timer.schedule(new TimerTask() {
                    @Override
                    public void run() {
                        err.getConnectionRequest().retry();
                    }
                }, 2000);
            } else {
                // this is the case of a server error
                // logs the error
                String errorLog = "REST ERROR\nURL:" + err.getConnectionRequest().getUrl()
                        + "\nMethod: " + err.getConnectionRequest().getHttpMethod()
                        + "\nResponse code: " + err.getConnectionRequest().getResponseCode();
                if (err.getConnectionRequest().getRequestBody() != null) {
                    errorLog += "\nRequest body: " + err.getConnectionRequest().getRequestBody();
                }
                if (err.getConnectionRequest().getResponseData() != null) {
                    errorLog += "\nResponse message: " + new String(err.getConnectionRequest().getResponseData());
                }
                if (err.getConnectionRequest().getResponseErrorMessage() != null) {
                    errorLog += "\nResponse error message: " + err.getConnectionRequest().getResponseErrorMessage();
                }
                Log.p(errorLog, Log.ERROR);

                Log.sendLogAsync();
                ToastBar.showErrorMessage("Server Error", 10000);
            }
        });
    }

}

Safe Uploads?

Implementing uploads with the same features (network error resistance and automatic resume) is more complex, because in this case we do not have a reference standard available by default on the most common servers.

Moreover, the possibility of partial uploads assumes that, after a network error, the server must keep the partially uploaded file and there are no ambiguities about which client has partially uploaded which file.

Applications such as Dropbox, Google Drive, OwnCloud and similar use specific internal standards. As far as I’m concerned, I’m almost completed deploying my own client-server solution to allow secure, network error-resistant with automatic resume uploads with Codename One and Spring Boot. This solution, however, is too specific to be included in the Codename One API and, anyway, I still have to do a lot of testing to make sure it works as it should. I’ll possibly publish a tutorial about it when it is finished.

Share this Post:

Posted by Francesco Galgani

Francesco loves the open-source philosophy and dreams a better world. He's a developer and a Codename One enthusiast, he blogs at www.informatica-libera.net.