How To Get Data From Google Analytics With Java

In the previous post, we discussed a strategy to ingest Google Analytics data and presented the Python code example (How To Get Data From Google Analytics With Python). Generally speaking, I prefer using Python code to ingest API data, however, every once in a while, we get requested to write a code in a compiled language like Java for whatever reason. So, here is GA data ingestion code example in Java.

For authentication, you need to download Google Analytics API Client Library for Java. The code below is based on the Google documentation here. The report extraction strategy is slightly different from the python code. This example allows you to specify the dimensions and metrics along side with other arguments while the previous example has multiple reports hard coded in the main method. With this strategy, you have a flexibility of changing the report dimensions and metrics without touching the source code. If you need to export multiple reports, you can execute the jar file with different arguments. The code will download data as json and convert to csv in a structured format ready to be uploaded into a relational database.

Execution
Complie the code into an executable jar file and run the command with arguments.

java -jar GA_Data_Export.jar <key dir e.g. ./OUA-Key.p12> <service email>
<json file path e.g. ./ga_export.json> <csv file path e.g. ./ga_export.csv>
<metrics e.g. ga:users,ga:sessions,ga:bounceRate>
<dimensions e.g. ga:channelGrouping,ga:date>
<start-date e.g. 7daysAgo or 2017-01-20>
<end-date e.g. today or 2017-12-25>

Code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
public class GoogleAnalyticsAuth {
   
    private static final String APPLICATION_NAME = "GA Data Ingestion";
    private static final JsonFactory JSON_FACTORY = GsonFactory.getDefaultInstance();
    private static String KEY_FILE_LOCATION = "";
    private static final HttpTransport HTTP_TRANSPORT = new NetHttpTransport();
    private static String apiEmail = "";
    private static String jsonFilePath = "";
    private static String csvFilePath="";

    public static void main(String[] args) {
       
        KEY_FILE_LOCATION = args[0];
        apiEmail = args[1];
        jsonFilePath = args[2];
        csvFilePath = args[3];
           
        String dimsMetric = args[4];
        String dimsDimension = args[5];
        String startDate = args[6];
        String endDate = args[7];
       
        try {
            Analytics analytics = initializeAnalytic();
            String profile = getFirstProfileId(analytics);
            System.out.println("First Profile Id is: " + profile);
           
            GaData result = getResults(analytics, profile, dimsMetric, dimsDimension, startDate, endDate);

            generateJsonFile(result, jsonFilePath);
            generateCsvFile(result, csvFilePath);
           
            // System.out.println(json.getString("rows"));
            // System.out.println(generateHeaderString(result));
           
        } catch (Exception e) {
            e.printStackTrace();
        }

    }
   
    // Get Data
    private static GaData getResults(Analytics analytics, String profileId,
            String dimsMetric, String dimsDimension,
            String startDate, String endDate) throws IOException {
        // Query the Core Reporting API for the number of sessions
        // String dimsDate = ",ga:date, ga:hour";      
       
        Get get = analytics.data().ga().get("ga:" + profileId, startDate, endDate, dimsMetric);
        get.setDimensions(dimsDimension);
        get.setMaxResults(10000);
       
        return get.execute();
      }
   
    private static String generateHeaderString(GaData results) {
       
        String columns = "";
        List<ColumnHeaders> header = results.getColumnHeaders();
        Iterator<ColumnHeaders> iter = header.iterator();
        while (iter.hasNext()) {
            columns += iter.next().getName().replace("ga:",  "") + ",";
        }
        return columns.substring(0, columns.length() - 1);
    }
   
    private static void generateCsvFile(GaData results, String csvFilePath) {
       
        try {
            // (1) Prepare files
            FileWriter fWriter = new FileWriter(csvFilePath);
            BufferedWriter bWriter = new BufferedWriter(fWriter);
            PrintWriter out = new PrintWriter(bWriter);
           
            // (2) Write header
            out.println(generateHeaderString(results));
           
            // (3) Get data from GaData object
            List<List<String>> rows = results.getRows();
            Iterator<List<String>> iter = rows.iterator();
            while (iter.hasNext()) {
                Iterator<String> iter2 = iter.next().iterator();
                String row = "";
                while (iter2.hasNext()) {
                    row += iter2.next() + ",";
                }
                out.println(row.substring(0, row.length() - 1));
               
                // System.out.println(row.substring(0, row.length() - 1));
            } // end of first while loop
           
            // (4) Close the file
            out.close();
            System.out.println("CSV File has been created.");
           
        } catch (IOException e) {
            System.out.println("Problem with csv file IO operation: " + e.toString());
        }
    }

    private static void generateJsonFile(GaData results, String filePath) throws JSONException {
        try {
            // (1) Prepare files
            FileWriter fWriter = new FileWriter(filePath);
            BufferedWriter bWriter = new BufferedWriter(fWriter);
            PrintWriter out = new PrintWriter(bWriter);
            // (2) Create JSON Object and write into the file with indentation
            JSONObject json = new JSONObject(results);
            out.println(json.toString(4));
            // (3) Close the file
            out.close();
            System.out.println("JSON File has been created.");
        } catch (IOException e) {
            System.out.println("Problem with JSON file IO operation: " + e.toString());
        }
    }
   
    // AUTHENTICATION
   
    /**
     * Initializes an Analytics service object.
     *
     * @return An authorized Analytics service object.
     * @throws IOException
     * @throws GeneralSecurityException
     */

    private static Analytics initializeAnalytic() throws GeneralSecurityException, IOException {
       
        HttpTransport httpTransport = GoogleNetHttpTransport.newTrustedTransport();
        GoogleCredential credential = new GoogleCredential.Builder()
                .setTransport(HTTP_TRANSPORT)
                .setJsonFactory(JSON_FACTORY)
                .setServiceAccountId(apiEmail)
                .setServiceAccountScopes(Arrays.asList(AnalyticsScopes.ANALYTICS_READONLY))
                .setServiceAccountPrivateKeyFromP12File(new File(KEY_FILE_LOCATION)).build();
        /* Using JSON file for secret key
        GoogleCredential credential = GoogleCredential.fromStream(new FileInputStream(KEY_FILE_LOCATION))
                                                        .createScoped(AnalyticsScopes.all());
        */

       
        return new Analytics.Builder(httpTransport, JSON_FACTORY, credential)
                .setApplicationName(APPLICATION_NAME).build();
    }
   
    private static String getFirstProfileId(Analytics analytics) throws IOException {
        // Get the first view (profile) ID for the authorized user.
        String profileId = null;

        // Query for the list of all accounts associated with the service account.
        Accounts accounts = analytics.management().accounts().list().execute();

        if (accounts.getItems().isEmpty()) {
          System.err.println("No accounts found");
        } else {
          String firstAccountId = accounts.getItems().get(0).getId();

          // Query for the list of properties associated with the first account.
          Webproperties properties = analytics.management().webproperties()
              .list(firstAccountId).execute();

          if (properties.getItems().isEmpty()) {
            System.err.println("No Webproperties found");
          } else {
            String firstWebpropertyId = properties.getItems().get(0).getId();

            // Query for the list views (profiles) associated with the property.
            Profiles profiles = analytics.management().profiles()
                .list(firstAccountId, firstWebpropertyId).execute();

            if (profiles.getItems().isEmpty()) {
              System.err.println("No views (profiles) found");
            } else {
              // Return the first (view) profile associated with the property.
              profileId = profiles.getItems().get(0).getId();
            }
          }
        }
        return profileId;
      }
}
Data Engineering
Sending XML Payload and Converting XML Response to JSON with Python

If you need to interact with a REST endpoint that takes a XML string as a payload and returns another XML string as a response, this is the quick guide if you want to use Python. If you want to do it with Node.js, you can check out the post …

Data Engineering
Sending XML Payload and Converting XML Response to JSON with Node.js

Here is the quick Node.js example of interacting with a rest API endpoint that takes XML string as a payload and return with XML string as response. Once we get the response, we will convert it to a JSON object. For this example, we will use the old-school QAS (Quick …

Data Engineering
Downloading All Public GitHub Gist Files

I used to use plug-ins to render code blocks for this blog. Yesterday, I decided to move all the code into GitHub Gist and inject them from there. Using a WordPress plugin to render code blocks can be problematic when update happens. Plugins might not be up to date. It …