Individual Assignment 2

From CS1635 Spring 2014
Jump to: navigation, search

Deadline: by 9:00AM on February 6, 2014

Contents

Overview

In this assignment you will build a handwriting/gesture recognition application that lets the users sketch English characters, numeric characters or even characters in a foreign language on their mobile device, send the captured result to a remote cloud service, and display the results from the remote server properly on the mobile device. This application can be treated as the basis of a mobile note taking application, a mobile handwriting dictionary, or a cloud based gesture interface.

The purpose of the assignment is to deepen your understanding of drawing and touch event-handling on the Android platform and to warm up your existing Java network programming (HTTP Post) skills.

You will submit three things:

  1. your source code
  2. your executable (Make sure your executable has all resources that it needs to run.)
  3. a brief description about major challenges you addressed or lessons learned from this assignment.

Design Requirements

Requirements:

  • Application:
    • Create an application that accepts and responds to touch events. (reference: Input Events)
    • Using the touch data, track and display single finger movements on a canvas (Please refer to chapter 9 of Programming Android for drawing on a canvas) (Note: when you visualize your finger traces, you can use straight lines (3 or 4 pixels wide) to connect dots collected in a single stroke)
    • User must be able to clear the writing area
    • User must be able to select the color for displaying ink traces (selecting between a few pre-defined colors is sufficient)
    • The application should have at least two activities


One way to satisfy the application functionality requirements is to create an application with two views: one of the canvas, and another of an options screen that lets the user change modes and adjust settings.

Extra credit: Up to 5 extra credit points are available if you design and implement more features, such as:

  • Saving and loading ink traces to device memory (see Data Storage.)
  • Implement a customized widget for ink capture and display.

Notes

Implementation

Client

First you need to get the user's input. You will need an OnTouchListener over the paint window to catch touch events.

There are a few ways to paint in Android. One way is to paint directly on a View object. If performance is an issue, i.e. for real-time graphics as is the case here, it is better to paint onto a Canvas object which gives the programmer more control and is usually faster. To do this, define a custom class which extends View and override its onDraw method. onDraw is where you do your drawing. onDraw will automatically be called with a Canvas object argument, so you don't need to extract the canvas from the View. In order to draw on the canvas, you will need to create one or more paint objects with the desired colors.

To put this custom View object in the hierarchy, create an instance of it in the main onCreate() method as you would for other Views, and then make it a child of a View or Layout widget that you have included in your XML layout file using View.addView. The new view will have "fill_parent" attributes by default, and so will fill available space which is probably what you want.

Because painting in Android (like most other GUI toolkits) is "smart", the display doesn't automatically change when you paint on the canvas. Instead you have to tell Android that the image really changed by calling View.invalidate().

This is by no means a brilliant design. You are free to design your own app provided that it has at least the functionality enumerated above.

Talking with the server/cloud

You do not need to create your own recognition service for this assignment, but you will need to encode your own handwriting data in the proper format, transmit the data to the correct cloud service access URL with proper protocol, and then interpret/display the response properly.

  • The communication protocol: The recognition service has two access points. One is http://cwritepad.appspot.com/reco/usen, which supports the recognition of English alphabetic and numeric characters. The other one is http://cwritepad.appspot.com/reco/gb2312, which supports the recognition of two languages, both English and Simplified Chinese. You will get full credit no matter which access point you choose for this assignment. If you choose the second one, it may be possible to build something more interesting (e.g. a foreign language learning application or a handwriting dictionary for a foreign language). You need to send an HTTP Post request to one of the above URLs to activate the service. The HTTP Post request will have two parameter pairs, the first one is the service key, in the format of "key=11773edfd643f813c18d82f56a8104ed", the second parameter is the actual ink data, in the format of "q=[30, 121, 82, 121, 121, 121, 255, 0, 121, 21, 120, 85, 120, 185, 255, 255]". You can use Java classes like HttpClient, HttpPost, and UrlEncodedFormEntity etc to talk with the server. (hint: no more than 20 lines of code are needed here)
  • The handwriting/ink encoding format: We use a very simple format to encode ink traces and send them to the server or log them locally. Valid coordinate range is [0, 254] for each axis, so please scale your ink coordinates linearly to [0, 254] before sending them to the server. (255, 0) means end of stroke, (255, 255) means end of encoding. For example, "[30, 121, 82, 121, 121, 121, 255, 0, 121, 21, 120, 85, 121, 115, 120, 185, 255, 255]" is the actual encoding of a "cross" (i.e. "+" symbol) with two strokes (there are three sample points in the first stroke, there are four sample points in the second stroke, there are usually dozens of sample points in each of your actual ink strokes).
  • Server response: The response from the server will be in plain text format (i.e. "text/plain" in HTTP header). For the first access point, the response will be a list of recognized candidates separated by the white space, e.g. "1 l L 2 ". For the second access point, the response will be a list of recognized candidates, with two bytes for each candidate. For English characters, a space holder (i.e. the white space) is appended to each character. Each Chinese character is represented with 2 bytes in GB2312 format. (Hint, you can use the Java function call "new String(buf, 0, len, "GB2312")" to decode the returned result and create a Java String properly).
  • Sample implementation:, We provide a working sample implementation in Python to let you test the ink encoding format and the server side communication protocol when you are creating your own mobile application. You may also test the cloud service in a web browser as a web based application.

Grading criteria

Up to 20 points will be given if your application compiles, runs, contains the functionality as detailed in the instructions.

Up to 5 extra points will be given if you implement additions that make the application more usable or more useful.

Submission Instructions

Create a Wiki Page for this assignment

Begin by creating a new wiki page for this assignment. Go to your user page that you created when you made your account. You can get to it by typing the following URL into your browser:

http://mips.lrdc.pitt.edu/cs1635-spring14/User:FirstName_LastName

Replace FirstName and LastName with your real first and last names. This will take you to the page you created for yourself when you created your wiki account. If you have trouble accessing this page, please check that you created your wiki account properly.

Edit your user page to add a link to a new wiki page for this assignment. The wiki syntax should look like this:

[[IndividualAssignment2-FirstName_LastName|Individual Assignment 2]]

Again replace FirstName and LastName with your name. Look at my user page for an example. Then click on the link and enter the information about your assignment. You should upload two files (described below) and in the wiki page, describe any extra credit functionality you implemented and want us to review.

Upload Project

  • Your submitted project must include the full source code as well as the executable of the working application and the one page challenges and lessons learned.
  • Create a zip file of your project tree (the top-level folder that includes the Eclipse project. Rename the zip file to apk2-firstname-lastname.zip (e.g., apk2-jingtao-wang.zip)
  • Upload the zip file to the IndividualAssignment2-FirstNameLastName page you just created:
    • Create a new file link like this: [[File:apk2-firstname-lastname.zip]]
    • Save the page, then click on the File link you just created to upload the zip file.

Add Link to Your Finished Assignment

One you are finished editing the page, add a link to it at the bottom of the page with your full name as the link text. The wiki syntax will look like this: *[[IndividualAssignment2-FirstName_LastName|FirstName LastName]]. Hit the edit button for the last section to see how I created the link for my name.

Links to Finished Assignments

Add your submission below this line: