torsdag 11 februari 2016

Android : Using HttpURLConnection instead of Apache httpClient

As of Android 6.0 Google has removed support for the Apache HTTP client.

Quote from http://developer.android.com/about/versions/marshmallow/android-6.0-changes.html

Apache HTTP Client Removal

Android 6.0 release removes support for the Apache HTTP client. If your app is using this client and targets Android 2.3 (API level 9) or higher, use the HttpURLConnection class instead. This API is more efficient because it reduces network use through transparent compression and response caching, and minimizes power consumption. To continue using the Apache HTTP APIs, you must first declare the following compile-time dependency in your build.gradle file:
android {
    useLibrary 'org.apache.http.legacy'
}

This post will help you with the basics of HttpURLConnection.
First example just loads a simple url, second sends a POST to automatically login.

Connecting to simple page is simple, lets create a helper class. Notice, these are just examples, exceptions are not handled, internet connectivity is not checked.

HttpWorker.java
package se.adanware.httptest.example;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.*;

public class HttpWorker {

    private URLConnection urlConnection;
    private URL myUrl;
    private String htmlData;


    public boolean connect(String url) throws IOException
    {
        myUrl = new URL(url);
        urlConnection = myUrl.openConnection();
        htmlData = convertStreamToString(urlConnection.getInputStream());
        return true;
    }


    private String convertStreamToString(InputStream is) throws IOException {

        if (is != null) {
            BufferedReader reader;
            StringBuilder data = new StringBuilder();
            try
            {
                reader = new BufferedReader(new InputStreamReader(is, "ISO-8859-1"));

                String inputLine;
                while ((inputLine = reader.readLine()) != null)
                {
                    data.append(inputLine + "\n");
                }
            }
            finally
            {

                is.close();
            }
            return data.toString();
        } else {
            return "";
        }
    }
}

Lets use it in an Activity. Remember we can't run in on the main thread, we'll create a simple AsyncTask to do the work, i'll make it as barebone as possible.

MainActivity.java
package se.adanware.httptest.example;

import android.os.AsyncTask;
import android.support.v7.app.AppCompatActivity;
import android.os.Bundle;
import android.util.Log;

public class MainActivity extends AppCompatActivity {
    private HttpWorker httpWorker;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        new ExampleTask().execute("https://stackoverflow.com/users/login?ssrc=head");
    }

    private class ExampleTask extends AsyncTask<String,Void,Void>
    {
        protected Void doInBackground(String... urls) {
            httpWorker = new HttpWorker();
            try {
                httpWorker.connect(urls[0]);
            }
            catch (Exception e)
            {
                // Just an example, we just swallow. : )
            }
        }

        protected void onProgressUpdate(Void... progress) {
            // Just an example.
        }

        protected void onPostExecute(Void result) {
            Log.d("HttpWorker-Example", "Set breakpoint somewhere to inspect htmldata.");
        }
    }
}

All right, we got the html data from StackOverflow login page. Lets say we don't have an API to work with and we like to do a POST to automatically login to parse some data.

Fiddler is the program to use to track all post values and post urls.

Their first login form post query is as follows :
isSignup=false&isLogin=true&isPassword=false&isAddLogin=false&hasCaptcha=false&fkey=loooong&ssrc=head&email=myemail@somewhere.com&password=mypassword&submitbutton=Log+in&oauthversion=&oauthserver=&openidusername=&openididentifier=

Login page will also make a second post with another query, it's about the same minus a few variables. Download Fiddler and you can track the whole login process.
So, fkey value need to be parsed but the rest is static, lets rewrite our HttpWorker class.

package se.adanware.httptest.example;

import android.net.Uri;

import javax.net.ssl.HttpsURLConnection;
import java.io.*;
import java.net.*;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

public class HttpWorker {

    private HttpsURLConnection urlConnection;
    private URL myUrl;
    private String htmlData;
    private String htmlData_MAINPAGE;

    private static String POST_URL_1ST = "https://stackoverflow.com/users/login-or-signup/validation/track";
    private static String POST_URL_2ND = "https://stackoverflow.com/users/login?ssrc=head";
    private static String URL_MAINPAGE = "https://stackoverflow.com";
    private String fkey_value;

    public HttpWorker()
    {
        CookieHandler.setDefault( new CookieManager( null, CookiePolicy.ACCEPT_ALL ) );
    }

    public boolean connect(String url) throws IOException
    {
        myUrl = new URL(url);
        urlConnection = (HttpsURLConnection) myUrl.openConnection();
        htmlData = convertStreamToString(urlConnection.getInputStream());
        fkey_value = parsefkeyValue(htmlData);
        return true;
    }

    public void login(String username, String password) throws IOException
    {
        URL loginPostURL = new URL(POST_URL_1ST);

        urlConnection = (HttpsURLConnection) loginPostURL.openConnection();
        // Simple to build query with Uri.Builder
        Uri.Builder builder = new Uri.Builder()
                .appendQueryParameter("isSignup", "false")
                .appendQueryParameter("isLogin", "true")
                .appendQueryParameter("isAddLogin", "false")
                .appendQueryParameter("hasCaptcha", "false")
                .appendQueryParameter("fkey", fkey_value)
                .appendQueryParameter("ssrc", "head")
                .appendQueryParameter("email", username)
                .appendQueryParameter("password", password)
                .appendQueryParameter("submitbutton", "Log in")
                .appendQueryParameter("oauth_version", "")
                .appendQueryParameter("oauth_server", "")
                .appendQueryParameter("openidusername", "")
                .appendQueryParameter("openididentifier", "");

        String query = builder.build().getEncodedQuery();
        urlConnection.setRequestMethod("POST");
        urlConnection.setRequestProperty("Content-Type","application/x-www-form-urlencoded");
        urlConnection.setDoInput(true);
        urlConnection.setDoOutput(true);
        sendPostParams(urlConnection.getOutputStream(), query);
        String LOGIN_STATUS = convertStreamToString(urlConnection.getInputStream());
        // Check the reply, do the repost if login sucessful.
        if(LOGIN_STATUS.contains("Login-OK"))
        {
            loginPostURL = new URL(POST_URL_2ND);
            urlConnection = (HttpsURLConnection) loginPostURL.openConnection();
            builder = new Uri.Builder()
                    .appendQueryParameter("fkey", fkey_value)
                    .appendQueryParameter("ssrc", "head")
                    .appendQueryParameter("email", username)
                    .appendQueryParameter("password", password)
                    .appendQueryParameter("oauth_version", "")
                    .appendQueryParameter("oauth_server", "")
                    .appendQueryParameter("openidusername", "")
                    .appendQueryParameter("openididentifier", "");

            urlConnection.setRequestMethod("POST");
            urlConnection.setRequestProperty("Content-Type","application/x-www-form-urlencoded");
            urlConnection.setDoInput(true);
            urlConnection.setDoOutput(true);
            sendPostParams(urlConnection.getOutputStream(), builder.build().getEncodedQuery());

            htmlData = convertStreamToString(urlConnection.getInputStream());
            URL mainPage = new URL(URL_MAINPAGE);
            urlConnection = (HttpsURLConnection) mainPage.openConnection();
            htmlData_MAINPAGE = convertStreamToString(urlConnection.getInputStream());
        } 

    }

    private void sendPostParams(OutputStream os, String params) throws IOException
    {
        BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(os, "UTF-8"));
        writer.write(params);
        writer.flush();
        writer.close();
        os.close();
    }

    private String convertStreamToString(InputStream is) throws IOException {

        if (is != null) {
            BufferedReader reader;
            StringBuilder data = new StringBuilder();
            try
            {
                reader = new BufferedReader(new InputStreamReader(is, "ISO-8859-1"));

                String inputLine;
                while ((inputLine = reader.readLine()) != null)
                {
                    data.append(inputLine + "\n");
                }
            }
            finally
            {

                is.close();
            }
            return data.toString();
        } else {
            return "";
        }
    }

    private String parsefkeyValue(String data)
    {
        Pattern myPattern = Pattern.compile("fkey\" value=\"([^\"]*)\"", Pattern.CASE_INSENSITIVE);
        Matcher ma = myPattern.matcher(data);
        if(ma.find())
            return ma.group(1);
        else
            return null;
    }
}

Let's try it out by adding httpWorker.login("my@mail.com, "myPassword"); after the connect method. Set a breakpoint after
htmlData_MAINPAGE = convertStreamToString(urlConnection.getInputStream()); 
to inspect the htmlData_MAINPAGE, you should see in the source a link to your user page etc.

It's a bit different from using Apaches HttpPost and the reference documentation doesn't explain it so well.

Hopefully it'll help someone!