(howto) Upload files to your Google bucket
In order to add data to your workspace, you need to upload your data files to the workspace bucket. You can upload data to your bucket through either the Google Cloud Platform Console, Google’s command line utility gsutil, or the FireCloud DataShuttle.
Watch the video demo or read more below.
Google Cloud Platform Console
- To access your workspace's bucket, navigate to your Workspace Summary tab, the Storage and Analysis section, and click on the bucket URL.
- Once you are in a bucket, you can click Upload Files or to download the file, click on its name.
First, install gsutil to your local computer. The Google Cloud SDK installation includes gsutil. To install Google Cloud SDK
You can run the following command using bash shells in your Terminal:
curl https://sdk.cloud.google.com | bash
Or download google-cloud-sdk.zip or google-cloud-sdk.tar.gz and unpack it. Note: The command is only supported in bash shells.
Restart your shell:
exec -l $SHELLor open a new bash shell in your Terminal.
gcloud initto authenticate, set up a default configuration, and clone the project's Git repository.
Before uploading data using gsutil, you can list buckets you have access to by running:
gsutil ls -p [project name]to list buckets for a specific project
- To upload data to a bucket run this command (you must be an Owner or Writer of the workspace to upload) :
gsutil cp [local file path] [bucket URL]
The bucket URL is the path to your file or folder in the Google Cloud SDK.
It will look like gs://[bucket name], e.g. gs://jntest10052015
or for folders within a bucket, gs://[bucket name]/[folder name], e.g., gs://jntest10052015/gene_files.
Note: To download data from a bucket, run
gsutil cp [bucket URL]/[file name], e.g., gs://jntest10052015/HCC1143.100_gene_250bp_pad.bam.
The FireCloud DataShuttle allows users to easily browse files, download and upload data directly between FireCloud workspaces & Google buckets and your local drives, and monitor the status of these transfers. It was developed to facilitate the work of researchers and project managers who transfer a high volume of files.