Survey - rev 116

The BETA-1 for ODK Survey is now available. This is a BETA release. It is not intended for production use.

This is an older version of our software.

The websites referenced in this documentation will generally have been updated to work with the newer software, and therefore may not function with this older release.

Where practical, we have provided zip files of the ApplicationDesigner environment used in creating those sites. See Survey and Tables Aux Files

Beta releases should never be used for production deployments. Beta releases are provided to gather bug reports (to make the application more robust) and for user feedback on the capabilities of the application. Updates may result in loss of data or incompatible changes in form designs.

We expect to release a second Beta with an incompatible database change as we align the database structures of ODK Survey, ODK Tables, and ODK Aggregate's new Tables storage subsystems.

We consider this Beta to be nearly code complete. We are considering changes to how the back button works (both the device back button and the back button within the app). We also expect to change the CSS styling of the widgets to create a flatter, more modern, look-and-feel.

This document covers the following topics.

  • Downloading and installing ODK Survey
  • Downloading Sample Forms
  • Opening a Form
  • Uploading Filled-in Submissions
  • Setting up a Form Development Environment.
  • Uploading Survey Forms to ODK Aggregate.

Downloading and Installing ODK Survey

To install the BETA, you must enable downloading from unknown sources by checking the checkbox beside Unknown sources on the Security screen of your Android device's Settings application.

Then, open your Android device's browser and download the ODK Survey 2.0 APK from here. Click on the ODK Survey APK link on that page and download the actual ODK Survey 2.0 BETA APK.

You should then be able to install that APK onto your device by choosing the Android Notification of the completion of the download. On some devices, you may need to open a file manager and double-tap on the downloaded file in the Downloads directory to trigger the install. On others, you may need to rename the file in the Downloads directory to end with .apk before double-tapping it to trigger the install.

Downloading Sample Forms

Sample forms have now been moved to a zip file available on our downloads page. Select the ODK Survey v2.0 Beta-1 Demo Files.zip from downloads. Unzip this file, and copy its contents to the /sdcard/opendatakit/ directory on your device.

ODK Survey looks at the contents of the /sdcard/opendatakit folder to locate user applications and their forms. User applications are sub-folders, and forms are nested within the tables folder within an application. When ODK Survey is opened, it defaults to using survey for the user application directory. The form definitions are therefore located under /sdcard/opendatakit/survey/tables/table_id/forms/form_id. table_id is the same as form_id unless otherwise specified; unlike with ODK Collect, with ODK Survey, you can have multiple forms editing and referencing the same data table.

Important Note: ODK Survey 2.0 cannot read or display the forms created for ODK Collect (e.g., those created via ODK Build, XLSForm or other form design tools).

The sample forms include:

  • Example Form -- a form with many examples of data entry widgets.
  • Household Member Survey -- a form used to gather information about household members. This is a sub-form of the Household Survey form, below. ODK Survey replaces the repeat group concept with sub-forms. This form (and its underlying data table) is a sub-form of the Household Survey form, below. From within the Household Survey you can navigate into this form to enter information about individuals in a household.
  • Household Survey -- a form used to gather information about a household. To operate correctly, this requires the Household Member Survey sub-form.
  • Initial refrigerator information -- together with Refrigerator information update form, this form demonstrates that ODK Survey can have two different form definitions that edit the same underlying data table. This functionality is incompatible with the 1.x data pipeline, but these two forms show what is possible with ODK Survey and the (not-yet-released) 2.0 data pipeline.
  • Refrigerator information update form -- see description for Initial refrigerator information above.
  • Select Examples -- a form with several examples of select widgets, including widgets that access data on Yahoo servers, and others that access CSV files for their choice lists. It also demonstrates the use of custom CSS styles to change the look of the form.
  • Two Part Validation Test -- a form that illustrates user-directed navigation of a form and the ability to validate different subsets of your gathered data at intermediate points within the form.

Opening a Form

After copying the demo files to the device, open ODK Survey. It will open to the Form Chooser screen. This is also reachable by choosing the 'open folder' icon on the title bar of the ODK Survey application. On that screen, select the form to open.

The form will now open. The application has been tested on Android 4.1 and higher devices. While it does work on 4.0 and earlier devices, not all features may function on those devices. On the original 2.2.3 Droid, the first load of the form will leave you with a white screen while the system reads and interprets the javascript framework files. This can take MINUTES.

Forms in ODK Survey are HTML. You can control the look-and-feel of the forms using CSS and add new prompt widgets by writing Javascript. To navigate forms using OpenDataKit's look-and-feel (i.e., the sample forms):

  • Tap on the name of the survey in the top left to access a pop-up menu of options.
  • Swipe or tap the 'back' or 'next' buttons in the top right of the form to navigate through the form.

To exit the form (and return to the form chooser screen), tap the device's back button (on newer devices, tap the back arrow icon at the bottom of the touch screen).

Note: The Common Javascript Framework form does not appear on the form chooser screen because it is the underlying set of HTML, CSS and javascript libraries that process and render the form definitions. It appears as a form for downloading because it is separately upgradable from the Java code and because of the way the ODK Aggregate legacy integration works. Once again, you must download that 'form' from ODK Aggregate in order for ODK Survey to work.

Uploading Filled-in Submissions

NOTE: uploading of filled-in submissions to the default http://opendatakit-2.appspot.com site may break as we release updates to ODK Survey. If they do not work against that site, set up your own site and try to publish to it. The form definition XML needed to be loaded into ODK Aggregate is the xforms.xml file found within each form directory in the Demo Files zip.

Once you have finalized a survey, you can upload the filled-in form to ODK Aggregate (or any OpenRosa 1.0 compatible server). This uses the same unidirectional submission mechanism as ODK Collect. To do so,

  1. tap the 'upload media (up arrow)' icon on the title bar of the ODK Survey application (you must finalize and exit the form first).
  2. tap on the form whose submissions you want to upload
  3. select the submissions to upload
  4. tap on Send Selected to upload the submissions.

Important Note: Once ODK Tables is released, the new ODK 2.0 submission mechanism will be introduced that allows both uploading and downloading of filled-in submissions, and the ability to update already-submitted records. The ability to update existing data is incompatible with the ODK 1.0 submission mechanism. ODK Survey allows you to revise already-submitted data, but, if you attempt to submit it again using the ODK 1.0 mechanisms, the updates will be rejected.

Setting up a Form Development Environment

You love the look, and want to jump into writing your own forms. The required steps are:

  • Get a Local Copy of the Framework
  • Set up a Local Web Server
  • Copy and Access Sample Forms
  • Develop New Forms

Get a Local Copy of the Javascript code

The Survey Javascript code for this release can be downloaded here.

Set up a Local Web Server

For Windows Users:

  • Install Apache HTTP server 2.4. This link may be helpful for the installation.

For Mac Users:

  • Configuring the Apache server depends on the Mac OS X version of the machine.  This link may be helpful for the configuration.

Copy and Access Sample Forms

Once Apache is installed and working properly, follow the steps below.

  1. Copy the files under opendatakit.survey-js/form-files/ to the htdocs/ directory of your Apache server in Windows or the /Library/WebServer/Documents/ directory on a Mac. This can be done by double clicking the copy.bat file for Windows or the copy-mac.command file on a Mac.  Both of these scripts are located under the opendatakit.survey-js/ directory. These copy scripts require that htdocs is in the default location.
  2. Chrome is the only web browser that can be used to run the JavaScript code.  Open the Chrome browser and type "http://localhost".
  3. You should now see the FORM/ and XLSX/ directories.  Click on the FORM/ directory.
  4. You should now see the framework/ and tables/ directories.  Click on the framework/ directory.  The ODK Survey UI should now be visible.
  5. You can now click one of the sample forms, and click 'Follow Link' to open that form in a new tab of the Chrome browser.

Develop New Forms

New forms must be properly placed under the 'form-files/tables' directory of the Javascript code. The general structure is:

form-files/tables/tableid/forms/formid/formid.xlsx

The tableid and formid are set on the settings tab of the spreadsheet. Note that the spreadsheet is required to be XLSX (the standardized, portable, Excel format).

See XLSXConverter documentation for how to develop your own form definitions.

Once you have defined you form, use your Chrome browser to navigate to http://localhost/XLSX (on Windows) or http://localhost/XLSX/WebContent/ on MacOSX. This displays the web page with the XLSXconverter. 

Choose or drag-and-drop the XLSX file into the drag area of this screen. This will run the XLSXConverter and display the compiled representation of it (the formDef.json) in the output window below. Save this into a file named formDef.json in the same directory as the XLSX file. Note: capitalization is significant. This file must be named formDef.json.

Once you have saved the file, copy the changes over to your local web server (see directions in earlier section).

Now, open your Chrome browser and point to http://localhost/FORM/framework/?#formPath=../tables/tableid/forms/formid/ This should open your form.

Important Note: Whenever you add new data fields to your form, you must Clear Browsing Data. We check every checkbox to clear absolutely everything. You can use an incognito page, but sub-form handling will not work if you do.

Uploading Survey Forms to ODK Aggregate

Once you have produced a form definition file (formDef.json) from the new XLSXConverter form definition, you can use the backward-compatibility function of ODK Survey to publish submissions into an existing ODK Aggregate 1.x server.

To do this, you must use a Windows PC.

  1. Create a forms directory on your system.
  2. Within this directory, create a sub-directory formid-media. The name of this sub-directory must end with -media.
  3. Within that sub-directory, place the formDef.json file produced by the XLSXConverter 2.0 process.
  4. If you have any additional files, such as custom css files, images, etc., copy them into this directory.
  5. If you have any nested directories of images or css files, create a zip file of each such directory, leave that in this sub-directory, and remove the nested directories. For example, the Select Examples form on the https://opendatakit-2.appspot.com site has many zip files holding the various JavaScript libraries and resources it uses. ODK Survey will in-place-unzip any .zip files after downloading them.
  6. Download (and save to a local file) https://github.com/opendatakit/app-designer/tree/ODK_Survey_2.0_beta_rev_116/form-files/convertToLegacy.hta. This is a scripting app that will only run on Windows systems.
  7. Double-click the convertToLegacy.hta app to launch it.
  8. Paste the full directory path to the forms directory into the Forms Directory: field and click Generate.
  9. The script will scan for sub-directories ending in -media and parse any formDef.json files contained within them, producing an ODK Aggregate 1.x-compatible XForms form definition file (.xml). These will be placed in the forms directory. So if you had example-media, an example.xml file would be generated in the forms directory.
  10. Now, run the ODK FormUploader 1.x tool available here to upload the form definition files (and their -media folders) to your ODK Aggregate 1.x instance.
  11. You can now run ODK Survey, download these forms from ODK Aggregate 1.x, and submit data into ODK Aggregate 1.x from ODK Survey.

NOTE: The converter script produces an XForms definition file (.xml) that is unusable by ODK Collect 1.x. It only contains enough information to support publishing of data into ODK Aggregate.