Firebase Cloud Storage

Firebase Cloud Storage provides a secure and reliable way for client apps to store and retrieve files located on the Google Cloud Storage servers using just a few lines of code. This chapter will introduce Firebase Cloud Storage covering storage references, metadata and the uploading and downloading of files. Security of the stored files will be the topic of the next chapter. 33.1 An Overview of Firebase Cloud Storage Firebase Cloud Storage allows files of any type to be stored using the Google Cloud Storage platform. Uploaded files are contained within Google Cloud Storage buckets. These storage buckets are also available to servers running within the Google Cloud ecosystem, providing the option to process the uploaded files via a server backend. By default the files for a client will be stored in a single bucket, though it is also possible to use multiple buckets. Behind the scenes, Firebase automatically handles all aspects of the transfer process, including retrying and resuming in the event of a loss of connectivity. In addition to the files themselves, Firebase also provides support for custom metadata to be stored along with the files. Security is implemented using a set of rules that are declared for the Firebase project in conjunction with Firebase Authentication. This ensures that the only users granted access to stored files are those who have been authorized to do so. The topic of cloud storage security rules is covered in the next chapter entitled A Guide to Firebase Cloud Storage Security Rules. Just like any other operating system filesystem, cloud-based files are stored using a hierarchical structure of nodes. These nodes equate to the files, directories and sub-directories of a traditional filesystem and are accessed using storage references. 33.2 Storage References Files are accessed from within a client app using a storage reference. Storage references are instances of the StorageReference class and are obtained by calling the getReference() method of the FirebaseStorage instance: StorageReference storageRef = FirebaseStorage.getInstance().getReference(); By default, a newly obtained storage reference instance points to the root node of the project’s default storage bucket (/). The child() method of the reference object is used to navigate deeper into the storage node hierarchy. The following code, for example, configures a reference to point to the /images/photos node of the storage bucket: photoRef = storageRef.child("/images/photos"); A storage reference may also be configured to reference a specific file: photoRef = storageRef.child("/images/photos/myPhoto.png"); In addition to the child() method, navigation is also possible using the getParent() and getRoot() methods of the storage reference instance. As the names suggest, the getRoot() method navigates to the top level of the storage node hierarchy, while the getParent() method returns the parent of the current node. Assuming a storage reference pointing at /images/photos, the following code would navigate to the /images/drawings node: photoRef = storageRef.child("/images/photos");

StorageReference secondRef = photoRef.getParent().child("drawings"); When working with the getParent() method, it is important to be aware that a filename in a path counts as a node level (referred to as the name node) when navigating in this way. Had the original reference been pointing to /images/photos/myPhoto.png, for example, it would have been necessary to call the getParent() method twice to navigate back as far as the /images node: photoRef = storageRef.child("/images/photos/myPhoto.png");

StorageReference secondRef = photoRef.getParent().getParent().child("drawings"); Though a client app may create as many storage reference objects as required, an existing storage reference may be reused as many times as needed. Once created and configured, references are used to upload, download, delete and update files stored using Firebase Cloud Storage. A range of methods is also available for obtaining information about a storage reference. 33.3 Storage Reference Properties The StorageReference class includes three additional methods that provide access to the properties of a reference. The getBucket() method returns the bucket within which the reference is pointing. This is returned as a string and will typically read as follows: <firebase-project-id>.appspot.com The getPath() method returns a string containing the full path to which the reference is pointing, for example: /images/drawings/houseplans.svg Finally, the getName() method returns the last component of the path. Assuming the above path, the method would return the filename as follows: houseplans.svg If the final component of the path is not a filename, the name of the last node is returned instead (in the case of the above example this would return drawings). 33.4 Uploading a File to Firebase Cloud Storage Firebase Cloud Storage provides three ways in which a file may be uploaded. The most basic involves the upload of an existing file located on the local device storage. To achieve this, the putFile() method of the storage reference instance is called, passing through as an argument the Uri of the file to be uploaded, for example: Uri fileUri = Uri.fromFile(new File("/path/to/local/file"));

photoRef = storageRef.child("/images/photos/myPhoto.png");

UploadTask uploadTask = photoRef.putFile(fileUri); When executed, the above code will upload the local file to the project’s cloud storage bucket and store it as myPhoto.png in the /images/photos node. 33.5 Uploading a File as a Stream An alternative upload option involves uploading the content of a file as an input stream. This involves the use of the putStream() method of the storage reference which also returns an UploadTask object: InputStream stream = new FileInputStream(new File("/path/to/local/file"));

photoRef = storageRef.child("/images/photos/myPhoto.png");

UploadTask uploadTask = photoRef.putStream(stream); 33.6 Uploading a File From Memory Data If the data to be contained in the file is currently stored in device memory, the putBytes() method of the storage reference will upload that data to a file in the cloud storage bucket. The putBytes() method takes as an argument a byte array (byte[]). The most likely use for this approach will be to upload an image that is currently residing in memory, perhaps within an ImageView object in a user interface. This image can be extracted from the ImageView object as a bitmap and then converted via byte stream to a byte array suitable for uploading to a cloud storage file: imageView.setDrawingCacheEnabled(true); imageView.buildDrawingCache(); Bitmap bmap = imageView.getDrawingCache(); ByteArrayOutputStream stream = new ByteArrayOutputStream(); bmap.compress(Bitmap.CompressFormat.JPEG, 100, stream); byte[] uploadData = stream.toByteArray(); photoRef = storageRef.child("/images/photos/myPhoto.png"); UploadTask uploadTask = photoRef.putBytes(data); 33.7 Monitoring and Managing the Upload When called, the putFile() method begins the upload process in the background and returns an UploadTask instance which can be used to monitor and manage the upload progress. The success or otherwise of the upload can be monitored by adding onSuccess and onFailure listeners to the UploadTask instance with corresponding callback methods as follows: uploadTask.addOnFailureListener(new OnFailureListener() {

   @Override
   public void onFailure(@NonNull Exception exception) {
       // Upload failed
   }

}).addOnSuccessListener(new OnSuccessListener<UploadTask.TaskSnapshot>() {

   @Override
   public void onSuccess(UploadTask.TaskSnapshot taskSnapshot) {
       // Upload succeeded
   }

}); The UploadTask object may also be used to pause, resume and cancel the upload via the pause(), resume(), and cancel() methods respectively. An onPaused listener may be added to receive notification when the upload is paused. Similarly, the callback method assigned to the onProgress listener will, if added, be called at regular intervals during the upload process. This can be useful for displaying the upload progress to the user. uploadTask.addOnProgressListener( new OnProgressListener<UploadTask.TaskSnapshot>() {

   @Override
   public void onProgress(UploadTask.TaskSnapshot taskSnapshot) {
       // Called at regular interval during upload
   }

}).addOnPausedListener( new OnPausedListener<UploadTask.TaskSnapshot>() {

   @Override
   public void onPaused(UploadTask.TaskSnapshot taskSnapshot) {
       // The upload has been paused.
   }

}); 33.8 Accessing Upload Task Snapshot Properties With the exception of the onFailure listener, each of these listener callback methods is passed an UploadTask.Snapshot object when called. This object contains the following methods which can be called to get information about the status of the upload: • getDownloadUrl() - If the upload was successful, this returns a URL string that can be used to download the file. • getError() - Returns an Exception object if the upload failed. • getBytesTransferred() - When called, this method returns the total number of bytes that had been transferred at the time the snapshot was created. • getTotalByteCount() - Returns the total number of bytes that will have been uploaded by the time the operation is complete. • getUploadSessionUri() - Returns a URI that can be used to continue the upload if a previous attempt was stopped before completion. The upload can be continued by making another putFile() method call, passing through this session Uri as the final argument. • getMetadata() - When called before the upload completes, this method returns the Metadata being uploaded along with the file. After the upload completes. The method returns the metadata held on the server. In both cases the metadata is returned in the form of a StorageMetadata object. • getTask() - Returns the UploadTask instance that was used to initiate the upload. • getStorage() - The StorageReference used to create the UploadTask. 33.9 Reading Metadata from an Uploaded File When a file is uploaded to cloud storage, a default set of metadata is included with the file. As mentioned above, the metadata for a stored file may be accessed by making a call to the getMetadata() method of an UploadTask.Snapshot object. The metadata for an uploaded file may also be accessed at any time via a call to the getMetadata() method of a storage reference object associated with the file. The following code, for example, requests the metadata for a stored file and adds two listeners to receive notification of the success or failure of the request: photoRef = storageRef.child("/images/photos/myPhoto.png");

photoRef.getMetadata().addOnSuccessListener( new OnSuccessListener<StorageMetadata>() {

           @Override
           public void onSuccess(StorageMetadata storageMetadata) {

// Metadata request succeeded

           }
       }).addOnFailureListener(new OnFailureListener() {
           @Override
           public void onFailure(@NonNull Exception exception) {

// Metadata request failed

           }
       }); 

If the request is successful, the metadata is passed to the onSuccess() callback method of the success listener in the form of a StorageMetadata object. The methods listed in Table 33 1 may then be called on this object to access specific property values:

Method Return Type Description getBucket() String The identifier of the Google Cloud Storage bucket in which the file resides. getCacheControl() String Provides control over the amount of time for which browsers are allowed to cache the stored file object. For example “max-age=2400”, “no-cache” etc. getContentDisposition() String Used to specify the information on how the file should be displayed when it is downloaded. Details on valid settings can be found at: https://tools.ietf.org/html/rfc6266 getContentEncoding() String The file encoding (for example ‘gzip’). getContentLanguage() String The language of the file content (for example ‘en’ or ‘ja’). getContentType() String The type of the file content (e.g. image/png). getCreationTimeMillis() long The date and time in milliseconds that the file was originally stored. getCustomMetadata(String key) String The metadata value associated with the specified key if custom metadata has been included with the file. getCustomMetadataKeys() <Set>String The keys for all custom metadata values assigned to the file. getDownloadUrl() Uri The URL by which the file can be downloaded. getDownloadUrls() <List>Uri A list containing the internal path Uris by which the file may be referenced. Unlike the getDownloadUrl() method, these are not HTTP URLs suitable for sharing with third parties. getGeneration() String An automatically generated string value indicating the current version of the saved file. getMd5Hash() String The MD5 hash of the stored file. getMetadataGeneration() String An automatically generated string value indicating the current version of the metadata associated with the file. getName() String The filename of the stored file. getPath() String The full path to the file within the cloud storage bucket. getReference() StorageReference The storage reference object associated with the stored file. getSizeBytes() long The size in bytes of the stored file getUpdatedTimeMillis() long The date and time in milliseconds that the stored file was last updated.

Table 33-1

The following onSuccess() callback method, for example, outputs the creation date and time, content type and size of an uploaded file:

photoRef.getMetadata().addOnSuccessListener(
		new OnSuccessListener<StorageMetadata>() {
    @Override
    public void onSuccess(StorageMetadata storageMetadata) {

    	long milliseconds = storageMetadata.getCreationTimeMillis();
        SimpleDateFormat formatter = 
	    new SimpleDateFormat("dd/MM/yyyy hh:mm:ss");

        String dateString = formatter.format(new Date(milliseconds));
        Log.i(TAG, "Created = " + dateString);
        Log.i(TAG, "Content type = " + storageMetadata.getContentType());
        Log.i(TAG, "File size = " + storageMetadata.getSizeBytes());
    }
.
.
});

The previous section explored the metadata properties that are included by default when a file is uploaded to the storage bucket. Firebase also offers the ability to change a subset of these default values using the StorageMetadata.Builder before the upload is performed. Once the StorageMetadata object has been created and configured, it is passed through as an argument to the appropriate put method when the upload is initiated. In the following code, the language and content type properties are changed within the StorageMetadata object before the file is uploaded:

StorageMetadata metadata = new StorageMetadata.Builder()
                .setContentType("image/jpg")
                .setContentLanguage("en")
                .build();

uploadTask = photoRef.putFile(fromUri, metadata);

The following methods are available for changing a metadata property before performing an upload using the above technique, all other StorageMetadata properties are read-only:

• setCacheControl()

• setContentType()

• setContentLanguage()

• setContentDisposition()

• setContentEncoding()

• setCustomMetadata()

The setCustomMetadata() method allows custom key-value pairs to be added to the file’s metadata as follows:

StorageMetadata metadata = new StorageMetadata.Builder()
                .setCustomMetadata("myKey1", "My Value One")
                .setCustomMetadata("myKey2", "My Value Two")                
		.build();

As outlined in Table 33-1, a list of custom keys within the metadata of a stored file may be obtained via a called to the getCustomMetadataKeys() method:

photoRef.getMetadata().addOnSuccessListener(
		new OnSuccessListener<StorageMetadata>() {
    @Override
    public void onSuccess(StorageMetadata storageMetadata) {

    	Set<String> customKeys = storageMetadata.getCustomMetadataKeys();

    	for (String s : customKeys) {
        	    Log.i(TAG, "key = " + s);
    	}
}

Similarly, the value assigned to a custom key can be accessed by passing the key through to the getCustomMetadata() method of the StorageMetadata object:

String customValue = storageMetadata.getCustomMetadata("myKey1");

The metadata assigned to an uploaded file can be changed by creating a new StorageMetadata object, customizing it with the values to be changed and then passing it to the updateMetadata() method of the storage reference object. This allows existing custom metadata properties to be changed and new custom metadata to be added. In addition, updates may be made to any standard properties that are not read-only:

StorageMetadata metadata = new StorageMetadata.Builder()
        .setContentType("image/png")
        .setContentLanguage("ja")
        .setCustomMetadata("myKey1", "A New Value")
        .setCustomMetadata("myKey3", "A New Key/Value")
        .build();

photoRef.updateMetadata(metadata)
        .addOnSuccessListener(new OnSuccessListener<StorageMetadata>() {
            @Override
            public void onSuccess(StorageMetadata storageMetadata) {
            }
        })
        .addOnFailureListener(new OnFailureListener() {
            @Override
            public void onFailure(@NonNull Exception exception) {
            }
        }); 

In addition to uploading and downloading files, it is also important to be able to delete files contained in a storage bucket. This involves a call to the delete() method of the storage reference object associated with the file to be deleted. Once again, the addition of listeners allows the success or otherwise of the deletion to be tracked:

photoRef.delete().addOnSuccessListener(new OnSuccessListener<Void>() {
    @Override
    public void onSuccess(Void aVoid) {
        // File deleted
    }
}).addOnFailureListener(new OnFailureListener() {
    @Override
    public void onFailure(@NonNull Exception exception) {
        // File deletion failed
    }
}); 

It is important to be prepared for the possibility that an upload process may get shutdown before the upload has completed. One option in this situation is to simply restart the upload from the beginning. A more efficient approach, however, is to resume the upload from the point at which the interruption occurred. This involves saving the current session Uri for the upload in persistent storage. In the event that the upload is interrupted, this session Uri can be retrieved and used to resume the upload.

Since this session Uri needs to be saved after the upload has started, the onProgress() callback method of the onProgress() listener method is the ideal location for this to take place. Each time the onProgress() method is called, it is passed an UploadTask.TaskSnapshot object. A call to the getUploadSessionUri() method of this object will return a String object containing the current session Uri. In the following example the session Uri is saved using shared preferences:

Context context = CurrentActivity.this;
SharedPreferences pref = getApplicationContext().getSharedPreferences(
				"StoragePrefs", context.MODE_PRIVATE);
SharedPreferences.Editor editor = pref.edit();
boolean saved = false;
.
.
uploadTask = photoRef.putFile(fromFile);

uploadTask.addOnProgressListener(new 
              OnProgressListener<UploadTask.TaskSnapshot>() {
      @Override
      public void onProgress(UploadTask.TaskSnapshot taskSnapshot) {
        Uri sessionUri = taskSnapshot.getUploadSessionUri();
        if (sessionUri != null && !saved) {
           saved = true;
           // Save Session Uri to persistent storage
           editor.putString("sessionUri", sessionUri.toString()).commit();
        }
      }
    }); 

When the client app is ready to resume the upload, the putFile() method is called, passing through the saved session Uri as the last argument as shown in the following code. Note that the session Uri does not retain any custom metadata from the original upload attempt so this needs to be reconstructed and included in the resumed upload:

// Restore saved session Uri from persistent storage
Uri sessionUri = Uri.parse(pref.getString("sessionUri", null));

StorageMetadata metadata = new StorageMetadata.Builder()
                .setContentType("image/jpg")
                .setContentLanguage("en")
                .build();

uploadTask = photoRef.putFile(fromFile, metadata, sessionUri);    

The Firebase console provides a screen where the files stored using cloud storage can be managed. Navigate to the console in a browser and select a Firebase project followed by the Storage option in the left-hand navigation panel. Within the Storage screen, select the Files tab as illustrated in Figure 33-1:

[[File:]]

Figure 33-1

To navigate through the folders in the bucket simply click on the links in the list. Clicking on the Upload File button allows a file to be uploaded into the currently selected folder. The folder button (marked A) to the right of the upload button allows a new folder be created in the current bucket.

To add or delete a storage bucket, click on the menu button (B) and select the Add bucket option from the menu. Note that it will be necessary to upgrade from the basic Spark Firebase plan to be able to add buckets.

To delete or download one or more items, select those items using the check boxes (C) and then click on the appropriate button in the toolbar:

[[File:]]

Figure 33-2

Firebase Cloud Storage allows files of any type to be stored in the cloud. Files are stored in cloud buckets and uploaded and downloaded using the FirebaseStorage instance. This instance is used to obtain a StorageReference object which is used to define the location of a file within cloud storage. In addition to the file, metadata is also included with the stored file providing information about the file such as file type, size and creation time. Custom metadata may also be included for storing app specific data relating to the file. In addition to working with stored files from within app code the Firebase console also provides an interface for navigating and managing the storage filesystem for a project.