Google Apps Script: Download mailbox of a domain user using Email Audit API in a Google Apps Domain


The Google Apps Email Audit API allows Google Apps administrators to audit a user’s email, email drafts, and archived chats. In addition, a domain administrator can retrieve account login information and download a user’s mailbox. This API only applies to Google Apps for Business, Education, and ISPs accounts. It is not used with a Google Apps or Gmail account not hosted by the Google Apps products.

Administrators can download mailbox accounts within their domain for audit purposes. To prepare a mailbox for export, the Email Audit service creates an encrypted copy of a user’s mailbox. When the export preparation is completed, the system returns the URLs to the encrypted mailbox files which, when downloaded and decrypted, are available in mbox format.

The downloading steps are:

  • Upload a public key
  • Create an export version of a user’s mailbox
  • Retrieve the mailbox download status
  • Upload a public key:
    The administrator provides a public encryption key for downloading mailboxes. The creation of this public key is only done once.

    Steps for generating public key :
    1. install GNUPG
    2. run gpg –gen-key –expert to generate a new key, selecting option 8 “RSA (set your own capabilities)” and toggling the sign capability.
    3. accept all default options to complete the key generation process.
    4. run gpg –armor –export to export the key. Please make sure you are only exporting the single key you just generated.
    5. visit http://www.motobit.com/util/base64-decoder-encoder.asp to base64-encode the key. Double check you are copying all the lines, including the header and not adding any extra line.

    After creation of public key, now we will upload public key.
    To upload the public key, start by creating an XML entry with the base64 encoded public key as shown in the example below:

    <atom:entry xmlns:atom='http://www.w3.org/2005/Atom' xmlns:apps='http://schemas.google.com/apps/2006'>
    <apps:property name="publicKey" value="the base64 Encoded Key"/>
    </atom:entry>
    

    Send an HTTP POST request to the ‘publickey’ feed URI in your Google Apps domain:

    POST https://apps-apis.google.com/a/feeds/compliance/audit/publickey/{domain name}
    

    If successful, the server returns a 201 CREATED status code.

    Let’s take code example for uploading a public key.

    function uploadPublicKey() {
       var publicKey=
       'LS0tLS1CRUdJTiBQR1AgUFVCTElDIEtFWSBCTE9DSy0tLS0tDQpWZXJzaW9uOiBHbnVQRyB2MS40'+
       'LjExIChHTlUvTGludXgpDQoNCm1RRU5CRkNPTnNrQkNBREpCWHQ5MEJLdGZqK3R3VnlJTDdCSU8z'+
       'amlrK05kRXJMQVhPSUVBSG5GUTVCV3pGM3gNCklHTXVTelAvOHZoenlkbm4rVW1YVVk3MDdEMnhJ'+
       'Y29PN055YXRDcXlvZi8vYitmemRyVkJqeTZES3J5alY5SFkNCjFPTkplcDhLdGNMbnZLaGVxcHBv'+
       'Mk5pK09OQUpvd2JBdTBtZnhCbUkvWFVsNkFCSlNOY2gyb0lCY0o5TVFYTDkNCkZkVWVLdmxIaUhL'+
       'b25QMm9VSU4vbm5kSitqOFV5NlgvUm01NEpOOUs2cWVkSkIweW5KdGZrczlEQWxhYlBmbWwNCnVj'+
       'R1grSHBwWDBqUkI5SFZ3cnRKQnNoQWlLTnRJanI1UTRHZ2MvdmxJVHZVWnJuN21ZajZFRFhkckFz'+
       'SGhCYUgNCmFnSDJMWEhMUnBFelpTWjVEWEFWNmd1MXNPd2VqWnR6Tnp1NUFCRUJBQUcwS2xKcGMy'+
       'aHBJQ2hMYm05c1pIVnoNCktTQThkbWxyWVhOQVpHRnRjMjl1ZEhKaGFXNXBibWN1WTI5dFBva0JP'+
       'QVFUQVFJQUlnVUNVSTQyeVFJYkRRWUwNCkNRZ0hBd0lHRlFnQ0NRb0xCQllDQXdFQ0hnRUNGNEFB'+
       'Q2drUThIMXVhNDdYNkVXZVlnZjlIZUlrRlVwRDNjR3ENClB2ajVQWjluTEZVb2dqM2dmR2ZMbXZz'+
       'RW5pUWJZNnFoSWFzVUxOOHFZZGxLL25vaDAxMGFzdXEwbUZqNFF1bWwNClI0bDBGOHdFZ2h3bU9j'+
       'dU5iczZQVUNoVlRIWVBhcTRPMUJWb0NqSFRKZU14M1NvUTBIbkM3TXdNZ3ZuS3pRSVYNClJsdVds'+
       'UWc4dERmZ3NGSktuL3JvSEdscWwwZ1QvNmNRdmdpcDhTcGw5MG1oMFVOQ2lEd3kzekg0cm9EbnEz'+
       'ZWoNCmhKVldQQ1ZDcU1iTjJhQUF3Zkg4OUpXSkhHOUJNR2ZVQm9Zbno3NFpYaWM5ekovUDVPanF1'+
       'Z1J1TXVyVTdiRnENCmNCQmRUbFcrTzNUdnhrT3RHQVRNcDBWWVIyUkJLREw0a2ZxcmdkR29rWmsw'+
       'T25pRmxXV1BYMmk1VllRZkVlbUQNCmtHcERacUNjc1E9PQ0KPS9RZ2UNCi0tLS0tRU5EIFBHUCBQ'+
       'VUJMSUMgS0VZIEJMT0NLLS0tLS0='
           
      var base='https://apps-apis.google.com/a/feeds/compliance/audit/'
      var fetchArgs=googleOAuth_('emailaudit',base)
      var rawXml="<atom:entry xmlns:atom='http://www.w3.org/2005/Atom' xmlns:apps='http://schemas.google.com/apps/2006'>"+
          "<apps:property name='publicKey' value='"+publicKey+"'/></atom:entry>"
      fetchArgs.payload=rawXml
      fetchArgs.contentType='application/atom+xml',
      fetchArgs.method='POST'
      var url='https://apps-apis.google.com/a/feeds/compliance/audit/publickey/[domainName]'
      var result=UrlFetchApp.fetch(url,fetchArgs)
    }
    
    

    Create an export version of a user’s mailbox:
    To prepare a copy of a user’s mailbox for export and downloading, use the Email Audit API’s export feed.
    Send a POST request to the export feed’s URI.

    POST https://apps-apis.google.com/a/feeds/compliance/audit/mail/export/example.com/liz
    
    <atom:entry xmlns:atom='http://www.w3.org/2005/Atom' xmlns:apps='http://schemas.google.com/apps/2006'>
       <apps:property name='beginDate' value='2009-07-01 04:30'/>
       <apps:property name='endDate' value='2009-08-30 20:00'/>
       <apps:property name='includeDeleted' value='false'/>
       <apps:property name='searchQuery' value='in:chat'/>
       <apps:property name='packageContent' value='FULL_MESSAGE'/>
    </atom:entry>
    

    If successful, the server returns a `201 CREATED` status code

    There are 5 properties in xml entry which are as:
    beginDate – The beginDate is the date for the first email included in the exported mailbox. This is an optional element. If you want all emails starting from when the account was created, do not enter a value for this field.
    endDate – The endDate is the date for the last email included in the exported mailbox. This is an optional element. If the endDate is not specified or an empty string, the exported emails go up to the current date.
    searchQuery – The mailbox is filtered using this searchQuery value and only the filtered search results will be available for download. This is an optional element.
    includeDeleted – The includeDeleted parameter determines whether or not deleted messages are included in the mailbox export file. This is an optional element
    packageContent – It determines whether the full email or the email’s header are used in the mailbox export file.
    FULL_MESSAGE — The full email text, including attachments, is copied to the export file. This is the default setting for the packageContent element.
    HEADER_ONLY — Only the email’s header is copied to the export file.

    Let’s take example code :

    function downloadMailBox(userID){
    try{
      var base='https://apps-apis.google.com/a/feeds/compliance/audit/'
      var fetchArgs=googleOAuth_('google',base)
      var rawXml='<atom:entry xmlns:atom="http://www.w3.org/2005/Atom" xmlns:apps="http://schemas.google.com/apps/2006">'+
          '<apps:property name="packageContent" value="FULL_MESSAGE"/></atom:entry>'
      fetchArgs.payload=rawXml
      fetchArgs.contentType='application/atom+xml',
      fetchArgs.method='POST'
      var urlForMailbox=base+'mail/export/[domainName]/[userName]?v=2&alt=json'
      var result=UrlFetchApp.fetch(urlForMailbox,fetchArgs).getContentText()
      var json=Utilities.jsonParse(result)
      var requestID=json.entry.apps$property[2].value
     }
    catch(e){
      downloadMailBox(userID)
      }
    }
    

    Note : In above code, downloadMailBox(userID) function again called in catch block. Because some times it gives time out error and control shifts towards catch block. In catch block, we are calling it again. So after retrying 2 or 3 times, it successfully executes.

    Retrieve the mailbox download status :
    Use the mailbox export request ID to get the status of the pending request. Once the mailbox is copied and prepared for export, the response returns a status of COMPLETED along with the list of encrypted mailbox files as HTTP URLs. Use this set of URLs to download the mailbox files.

    To retrieve status details for a mailbox prepared for export, send an HTTP GET request with the mailbox’s requestId to the export feed’s URI

    GET https://apps-apis.google.com/a/feeds/compliance/audit/mail/export/{domain name}/{source user name}/{mailbox requestId}
    

    These are the possible values of status :
    PENDING – The request is being processed.
    ERROR – The request failed due to some error.
    COMPLETED – The request has been processed completely and the encrypted mailbox files are ready for download.
    MARKED_DELETE – The request is marked for deletion next time the Google cleanup job runs.
    DELETED – The mailbox files were successfully deleted by the admin
    EXPIRED – The mailbox files were deleted by Google after the 3 week retention limit.

    Lets take a code example

    function checkingStatus(){
        var base='https://apps-apis.google.com/a/feeds/compliance/audit/'
        var requestID="resuest Id"
        var urlForCheckingStatus=
        base+'mail/export/[domainName]/[userName]/'+requestID+'?v=2&alt=json'
        var fetchArgs=googleOAuth_('google',base)
        fetchArgs.method='GET'
        fetchArgs.contentType='application/atom+xml'
        var result=UrlFetchApp.fetch(urlForCheckingStatus,fetchArgs).getContentText()
        var json=Utilities.jsonParse(result)
        var status=json.entry.apps$property[1].value
        
        if(status=="PENDING"){Logger.log('pending')}
        else if(status="COMPLETED"){
            var downloadURL=json.entry.apps$property[5].value
            // Do whatever you want with mailbox URL....
           }
        else if(status="ERROR"){
           Logger.log("error")
        }
     }
    
    function googleOAuth_(name,scope) {
      var oAuthConfig = UrlFetchApp.addOAuthService(name);
      oAuthConfig.setRequestTokenUrl("https://www.google.com/accounts/OAuthGetRequestToken?scope="+scope);
      oAuthConfig.setAuthorizationUrl("https://www.google.com/accounts/OAuthAuthorizeToken");
      oAuthConfig.setAccessTokenUrl("https://www.google.com/accounts/OAuthGetAccessToken");
      oAuthConfig.setConsumerKey(consumerKey);
      oAuthConfig.setConsumerSecret(consumerSecret);
      return {oAuthServiceName:name, oAuthUseToken:"always"};
    }
    

    To learn more about Email Audit API, see documentation

    About Rishi Khandelwal

    Software Consultant at Knoldus Software LLP, New Delhi, India having 3+ years industry experience. He has working experience in various technologies such as Scala, Java, Play, Akka, Lift Web, Backbone.js, html5, javascript, Less, Amazon EC2, WebRTC, SBT
    This entry was posted in Web and tagged , . Bookmark the permalink.

    3 Responses to Google Apps Script: Download mailbox of a domain user using Email Audit API in a Google Apps Domain

    1. Hello,
      I’ve successfully followed the tutorial but I have provided authentication failed to load the public key, but only for that function yet for all monitors an account authentication is successful.
      In both functions use the same form but I login via oauth to load the public key fails.

      function cargarClavePublica(){
      var dominio = “demotiga2.com”;

      var base=’https://apps-apis.google.com/a/feeds/compliance/audit/’;
      var xmlRaw=
      ”+
      ”+
      ”;

      var name = “audit”;
      var fetchArgs = {oAuthServiceName:name, oAuthUseToken:’always’, method:’POST’, payload:xmlRaw, contentType:’application/atom+xml’, muteHttpExceptions: true};

      var oAuthConfig = UrlFetchApp.addOAuthService(name);
      oAuthConfig.setRequestTokenUrl(“https://www.google.com/accounts/OAuthGetRequestToken?scope=”+base);
      oAuthConfig.setAuthorizationUrl(“https://www.google.com/accounts/OAuthAuthorizeToken”);
      oAuthConfig.setAccessTokenUrl(“https://www.google.com/accounts/OAuthGetAccessToken”);
      oAuthConfig.setConsumerKey(KEY);
      oAuthConfig.setConsumerSecret(SECRET);

      var url = base+’publickey/’+dominio;

      Logger.log(url);

      var urlFetch;
      var status;
      try{
      urlFetch = UrlFetchApp.fetch(url,fetchArgs);
      status = urlFetch.getResponseCode();
      Logger.log(status);
      }
      catch(e){
      Logger.log(“No se pudo cargar la clave pública : ” + e.message);
      }
      return status;
      }

    2. function cargarClavePublica(){
      var dominio = “demotiga2.com”;

      var base=’https://apps-apis.google.com/a/feeds/compliance/audit/’;
      var xmlRaw=
      ”+
      ”+
      ”;

      var name = “audit”;
      var fetchArgs = {oAuthServiceName:name, oAuthUseToken:’always’, method:’POST’, payload:xmlRaw, contentType:’application/atom+xml’, muteHttpExceptions: true};

      var oAuthConfig = UrlFetchApp.addOAuthService(name);
      oAuthConfig.setRequestTokenUrl(“https://www.google.com/accounts/OAuthGetRequestToken?scope=”+base);
      oAuthConfig.setAuthorizationUrl(“https://www.google.com/accounts/OAuthAuthorizeToken”);
      oAuthConfig.setAccessTokenUrl(“https://www.google.com/accounts/OAuthGetAccessToken”);
      oAuthConfig.setConsumerKey(“demotiga2.com”);
      oAuthConfig.setConsumerSecret(“F3FMHUED68LImercuczNnYME”);

      var url = base+’publickey/’+dominio;

      Logger.log(url);

      var urlFetch;
      var status;
      try{
      urlFetch = UrlFetchApp.fetch(url,fetchArgs);
      status = urlFetch.getResponseCode();
      Logger.log(status);
      }
      catch(e){
      Logger.log(“No se pudo cargar la clave pública : ” + e.message);
      }
      return status;
      }

    3. var xmlRaw= “”+
      “”+
      “”;

    Leave a Reply

    Fill in your details below or click an icon to log in:

    WordPress.com Logo

    You are commenting using your WordPress.com account. Log Out / Change )

    Twitter picture

    You are commenting using your Twitter account. Log Out / Change )

    Facebook photo

    You are commenting using your Facebook account. Log Out / Change )

    Google+ photo

    You are commenting using your Google+ account. Log Out / Change )

    Connecting to %s