Google Apps Script : To store data on google Cloud Sorage


Google Cloud Storage is a new service for developers to store and access data in Google’s cloud. It offers developers direct access to Google’s scalable storage and networking infrastructure as well as powerful authentication and data sharing mechanisms. Developers can store objects of any size and manage access to their data on an individual or group basis using Google Cloud Storage.

Google Cloud Storage is accessible through its REST API or using any of the available Google Cloud Storage tools, including:

Google Storage Manager – A graphical user interface with drag-and-drop features to manage your buckets and objects. https://developers.google.com/storage/docs/gsmanager

gsutil – A command line tool that lets you perform basic and advanced tasks for Google Cloud Storage. https://developers.google.com/storage/docs/gsutil

You can activate Google Cloud Storage through the Google APIs Console. If you have never used the Google APIs console, see the Google APIs Console help guide. https://developers.google.com/console/help/#ApiConsole

API Overview :
The Google Cloud Storage API is a RESTful interface that lets you programmatically manage data on Google Cloud Storage. As a RESTful API, the Google Cloud Storage API relies on method information and scoping information to define the operations you want to perform. You specify the method information with standard HTTP methods, such as DELETE, GET, HEAD, and PUT. And you specify the scoping information with a publicly-accessible endpoint (URI) and various scoping parameters. In this case, the primary scoping parameter is a path to a resource, which consists of an object name (file name) and bucket name (container name). Used together, the object name, bucket name, and public URI create a unique URL to a given resource—a resource on which you can perform operations with HTTP methods. You can further scope your operations by using HTTP headers and query string parameters.

The Google Cloud Storage API supports the following HTTP methods:

GET Service—lists all of the buckets that you own.
PUT Bucket—creates a bucket and changes the permissions on a bucket.
GET Bucket—lists the contents of a bucket or retrieves the ACLs that are applied to a bucket.
DELETE Bucket—deletes an empty bucket.
GET Object—downloads an object or retrieves the ACLs that are applied to an object.
PUT Object—uploads an object or applies new ACLs to an object.
DELETE Object—deletes an object.
HEAD Object—lists the metadata for an object.
POST Object—uploads an object by using HTML forms.

Authentication :
Most of the operations you perform with the Google Cloud Storage API must be authenticated. The only exceptions are operations on objects that allow anonymous access. Objects are anonymously accessible if the AllUsers group has READ permission. The AllUsers group includes anyone on the Internet.

OAuth 2.0 Authentication :

Google recommends OAuth 2.0 authentication for interacting with the Google Cloud Storage API. OAuth 2.0 authentication eliminates the need to provide sensitive account information, such as a username and password, to any applications that need access to your data. Instead, you can use the OAuth 2.0 protocol to obtain and give out OAuth tokens. OAuth tokens authenticate tools and applications to access Google Cloud Storage API on your behalf and also provides the ability to restrict access using scopes. You can authorize different applications with separate tokens, and revoke tokens individually, if necessary.

Read here more about authentication: https://developers.google.com/storage/docs/authentication

Google Cloud Storage API OAuth scopes are as follows:

read-only :Only allows access to read data, including listing buckets.
Scope URL :https://www.googleapis.com/auth/devstorage.read_only
read-write : Allows access to read and change data.
Scope URL https://www.googleapis.com/auth/devstorage.read_write
full-control : Allows full control over data, including the ability to modify ACLs.
Scope URL https://www.googleapis.com/auth/devstorage.full_control

Access Control :
There are two ways to control access to Google Cloud Storage objects and buckets:

1. Access Control Lists (ACLs), which uses Google accounts and provides longer term access. https://developers.google.com/storage/docs/accesscontrol#About-Access-Control-Lists
2. Signed URLs (Query String Authentication), which does not use Google accounts, but provides “valet-key” type access for a limited time. https://developers.google.com/storage/docs/accesscontrol#Signed-URLs

These are not mutually exclusive. You can use ACLs to protect buckets and objects, while at the same time providing signed URLs to users so they can access those resources, bypassing the ACL mechanism.

Here is an example of storing documents of a domain user on Google Cloud Storage. This script will be run at admin account of Google Apps Domain.

var consumerKey="yourDomain.com"
var domainName='yourDomain.c0m'
var consumerSecret="domainSecretkey"
var bucketNameOnGCS="bucketName"

function uploadDocuments() {
var userID='user@yourDomain.com'
var scope = 'https://docs.google.com/feeds/';
var fetchArgs = googleOAuth_('docs', scope);
fetchArgs.method= "GET"
var url = scope + userID+'/private/full?v=3&alt=json';

 // Getting Student's All Documents
var urlFetch = UrlFetchApp.fetch(url, fetchArgs) 
var json=Utilities.jsonParse(urlFetch.getContentText())
var entry = json.feed.entry;
for(var i in entry){
try{
var tempDoc = {};
var fileName={};
for(var j in entry[i]){ 
tempDoc.src=entry[i].content.src
fileName.id=entry[i].gd$resourceId.$t.split(":")[1]
}

// File_Name of Documents
var StudentfileName=DocsList.getFileById(fileName.id).getName()

// Exporting All Documets to pdf Format
var url=tempDoc.src+"&exportFormat=pdf&format=pdf"

// Getting document's content
var data=UrlFetchApp.fetch(url,fetchArgs)
var content=data.getContent()

var username=userID.split("@")[0]
var folderName=username+"_Documents"

// storing document in GCS
var url='https://storage.googleapis.com/'+bucketNameOnGCS+'/'+folderName+'/'+StudentfileName
storeIntoGCS(content,url)
}catch(e){}
}
}

// Storing data in GCS
function storeIntoGCS(content,url){
var base="https://www.googleapis.com/auth/devstorage.full_control"
var fetchArgs=googleOAuth_('provisioning',base)
fetchArgs.payload=content
fetchArgs.method='PUT'
fetchArgs.contentType="images/jpeg"
fetchArgs.host=bucketNameOnGCS+'.storage.googleapis.com'
var result=UrlFetchApp.fetch(url,fetchArgs)
}

// Oauth Authorization
function googleOAuth_(name,scope) {
var oAuthConfig = UrlFetchApp.addOAuthService(name);
oAuthConfig.setRequestTokenUrl("https://www.google.com/accounts/OAuthGetRequestToken?scope="+scope);
oAuthConfig.setAuthorizationUrl("https://www.google.com/accounts/OAuthAuthorizeToken");
oAuthConfig.setAccessTokenUrl("https://www.google.com/accounts/OAuthGetAccessToken");
oAuthConfig.setConsumerKey(consumerKey);
oAuthConfig.setConsumerSecret(consumerSecret);
return {oAuthServiceName:name, oAuthUseToken:"always"};
}

If we want to store the data as public on Google Cloud Storage then just add following content in storeIntoGCS(content,url) :

var header={'x-goog-acl':'public-read'}
fetchArgs.headers=header

About Rishi Khandelwal

Software Consultant at Knoldus Software LLP, New Delhi, India having 3+ years industry experience. He has working experience in various technologies such as Scala, Java, Play, Akka, Lift Web, Backbone.js, html5, javascript, Less, Amazon EC2, WebRTC, SBT
This entry was posted in Web and tagged , . Bookmark the permalink.

2 Responses to Google Apps Script : To store data on google Cloud Sorage

  1. prem says:

    Hi here what is mean by scope url

  2. prem says:

    and what is mean by RequestTokenUrl,AuthorizationUrl and AccessTokenUrl

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s