Recently I was working on Java application to scrap Reddit data using their api, although performing read-only operations (such as reading listing data) doesn’t require authentication but after reading a comment on StackOverflow I decided to use send authenticated request even if it isn’t necessary. Comment by the way says that for requests from robots/automatic scrappers without a reddit_session id, reddit returns limited amount of data and some time data is inconsistent too.
Anyways to get started first step is to send HTTP POST request to http://www.reddit.com/api/login along with required parameters ‘user’ and ‘passwd’ for username and password. If username and password combination matches reddit server will return authentication cookie containing reddit_session along with other different stuff like domain, path etc…
Code here show how to make an HTTP POST request using Apache HttpClient, read and store cookie for further use:
public class RedditClient {
private List<Cookie> cookies = null;
public boolean Login() throws ClientProtocolException, IOException {
List<NameValuePair> loginParams = new ArrayList<NameValuePair>();
loginParams.add(new BasicNameValuePair("api_type", "json"));
//Set values of form fields
loginParams.add(new BasicNameValuePair("user", userName));
loginParams.add(new BasicNameValuePair("passwd", password));
UrlEncodedFormEntity entity = new UrlEncodedFormEntity(loginParams, "UTF-8");
HttpPost httpPost = new HttpPost("https://ssl.reddit.com/api/login");
httpPost.setEntity(entity);
DefaultHttpClient httpClient = new DefaultHttpClient();
HttpResponse httpResponse = httpClient.execute(httpPost);
//Save cookie in List of Cookies so we can use it later
cookies = httpClient.getCookieStore().getCookies();
httpClient.getConnectionManager().shutdown();
return false;
}
}
Now in order to make a subsequent request which require authentication for example submitting a new post or posting a comment we have to pass attach this cookie with the request; Otherwise server will respond with an error.
Below is how to make a subsequent request, in this case a request to retrieve top 100 post from funny category, along with cookie.
public class RedditClient {
....
public void readListing() {
URI feedUrl = new URI("http://www.reddit.com/r/funny/top/?sort=top&t=month");
DefaultHttpClient httpClient = new DefaultHttpClient();
HttpGet httpGet = new HttpGet(feedUrl);
//Check if cookie list is not empty
//attach it with current request
if(this.cookies != null) {
for(Cookie c : this.cookies)
httpClient.getCookieStore().addCookie(c);
}
HttpResponse response = httpClient.execute(httpGet);
String strResponse = "";
BufferedReader rd = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
String line = "";
while ((line = rd.readLine()) != null) {
strResponse += line;
}
httpClient.getConnectionManager().shutdown();
}
}
In order to check if our request was treated as a trusted request look for the value of ‘modhash’ in response it should be a long encrypted string, for unauthenticated requests it’s always empty.
{"kind": "Listing", "data": {"modhash": "bjxvddjbxaac5b494bc70000000000000000000000",........ }
And finally putting it all together:
public class RedditClient {
private List<Cookie> cookies = null;
public boolean Login() throws ClientProtocolException, IOException {
List<NameValuePair> loginParams = new ArrayList<NameValuePair>();
loginParams.add(new BasicNameValuePair("api_type", "json"));
//Set values of form fields
loginParams.add(new BasicNameValuePair("user", userName));
loginParams.add(new BasicNameValuePair("passwd", password));
UrlEncodedFormEntity entity = new UrlEncodedFormEntity(loginParams, "UTF-8");
HttpPost httpPost = new HttpPost("https://ssl.reddit.com/api/login");
httpPost.setEntity(entity);
DefaultHttpClient httpClient = new DefaultHttpClient();
HttpResponse httpResponse = httpClient.execute(httpPost);
//Save cookie in List of Cookies so we can use it later
cookies = httpClient.getCookieStore().getCookies();
httpClient.getConnectionManager().shutdown();
return false;
}
public void readListing() {
URI feedUrl = new URI("http://www.reddit.com/r/funny/top/?sort=top&t=month");
DefaultHttpClient httpClient = new DefaultHttpClient();
HttpGet httpGet = new HttpGet(feedUrl);
//Check if cookie list is not empty
//attach it with current request
if(this.cookies != null) {
for(Cookie c : this.cookies)
httpClient.getCookieStore().addCookie(c);
}
HttpResponse response = httpClient.execute(httpGet);
String strResponse = "";
BufferedReader rd = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
String line = "";
while ((line = rd.readLine()) != null) {
strResponse += line;
}
httpClient.getConnectionManager().shutdown();
}
}
Recent Comments