February 2015 - rev 126

The documentation on this page corresponds to rev 126 (February 2015) of the tools suite. See Older Versions for documentation on earlier releases of this suite.

This is an older version of our software.The websites referenced in this documentation will generally have been updated to work with the newer software, and therefore may not function with this older release.

Where practical, we have provided zip files of the ApplicationDesigner environment used in creating those sites. See Survey and Tables Aux Files

Release Designations

The ODK 2.0 tools are in various stages of Alpha and Beta release.

  • Alpha software does not have all features, and is more likely to have significant reductions or additions of functionality. Alpha releases are provided to gather user feedback on the usability and capabilities of the application, as well as bug reports (to make the application more robust). Updates may result in loss of data or incompatible changes in form designs.
  • Beta software does not have all features, but is less likely to have significant reductions or alterations in functionality. Beta releases are provided to gather user feedback on the usability and capabilities of the application, as well as bug reports (to make the application more robust). Updates may result in loss of data or incompatible changes in form designs.
  • Alpha and Beta releases can be used at their current set of capabilities, but the ODK core team cannot provide support for deployments using them or guarantee a migration path into future releases. If you use these, you should plan to use them as-is, without expecting upgrades to newer versions of the tools. The ODK 2.0 tools have mechanisms to extract your data as CSV files from these tools, and to re-import data via CSV files. This provides a migration path to newer releases, but will generally require manual actions on your part.
  • Release Candidate software does not have support for deployments, but the ODK core team does guarantee a migration path (possibly with many tedious manual steps) into future releases.
  • Production software has been used in several deployments and there is some level of confidence that the software is stable. As with any software, there is no guarantee of being bug-free.

Non-technical users may find the usage descriptions difficult to follow and there may be a considerable number of manual steps needed to utilize the tools. As we progress from Alpha to Beta to Release Candidate and Production versions of the tools, they should become easier to use.

Overview

The next-generation ODK tools (the ODK 2.0 Tools Suite) are intended to address several limitations of the existing ODK Collect / ODK Aggregate / ODK Briefcase data collection workflow. These are:

  • Fully customizable layout of prompts on the Android device. The 2.0 tools use HTML, Javascript and CSS to specify the layout of nearly all the screens viewed by the data collectors. This enables individuals and organizations with basic web development skills to modify and customize the appearance of their surveys and workflow. At the same time, we retain the easy-to-use spreadsheet-based definition of the survey questions (however, this XLSXConverter mechanism is not cross-compatible with XLSForm).
  • More flexible, user-directed, navigation of a survey. The 2.0 tools do not impose a strict sequential advancement through a form like ODK Collect; form designers can allow users to traverse a form in any order, yet impose validation of collected data prior to traversing into subsequent steps in a workflow.
  • Improved treatment of repeat-groups. In the 2.0 tools, we have eliminated the concept of a repeat-group. In its place, we provide prompts that enable you to open and edit other surveys with links back to the originating survey (if desired). These prompts can describe a sub-form (nested) relationship among the surveys (e.g., household and household-member) or they can represent arbitrary relational linkages across your data (e.g., tea-houses and tea-types).
  • Bi-directional synchronization of data across devices. The ODK 2.0 tools support the collaborative sharing of survey data across devices, and the updating and submission of changes to previously-collected data (i.e., follow-up surveys) via a bi-directional synchronization protocol; this contrasts with the uni-directional device-to-server submission pathway of ODK Collect / ODK Aggregate / ODK Briefcase.
  • Data curation and visualization on the device. ODK Tables gives organizations the ability to investigate and visualize entire datasets directly on the Android devices through graphical and non-graphical displays and through filtered views.
  • Eliminate the need to fork the underlying Java codebase. The ODK 2.0 tools include an Application Packager that will eliminate the need for an organization to fork and maintain their own versions of the ODK Survey or ODK Tables applications in order to create their own branded and controlled app.

The 2.0 Tools Suite consists of:

  1. ODK Application Designer 2.0 rev 126- a design environment for creating, customizing and previewing your forms and data curation and visualization applications.
  2. ODK Application Packager 2.0 (future) - a tool to create your own signed APK (suitable for hosting on the Google Play Store) containing your application files such that your organization can have its own branded presence while leveraging the core ODK 2.0 tools without the risk of interaction between your application and those from other organizations.
  3. ODK Survey v2.0 rev 126 - a data collection application based upon HTML, CSS and Javascript.
  4. ODK Tables v2.0 rev 126 - a data curation and visualization application running on your device.
  5. ODK Sync v2.0 rev 126 - a data synchronization and access service used by both ODK Survey and ODK Tables to perform bi-directional data synchronization with a compatible web server (ODK Aggregate v1.4.4), and through that mechanism, share data across devices.
  6. ODK Aggregate Tables Extensions v2.0 rev 126 - enhancements to support bi-directional data synchronization across disconnected devices.

Trying It Out

First, see the ODK Survey and ODK Tables pages. They cover setting up these tools (and ODK Sync) and downloading and exploring demonstration forms and applications built upon these tools. These demonstration applications will give you a good sense of the flexibility and breadth of capabilities of the two tools.

Next, see the Getting Started Guide to understand the process for revising and developing your own forms. That guide will walk you through modifying the Geotagger demo app to add an additional field to it.

 

User-contributed Documentation

If you wish to contribute documentation or tutorials for the new tools, a Google doc has been started to hold user-contributed documentation here.

If you feel a section in that document is stable enough and complete enough to move onto the opendatakit.org site, please contact us to coordinate the move.

 

Limitations and Expected Changes

If you hit the back button before any of the applications complete their initial screen, your device can become non-responsive and display a black screen. This is due to a database lock not being released as you switch among the different tools (ODK Tables, ODK Survey and ODK Sync). In this case, pressing the home button will present you with a system prompt asking you whether to exit the application or wait for it. Answer OK (to exit the application); this will force close the application. You will then need to go to the Settings / Apps screen and Force Close the other ODK tools in order to release the database lock.

Within ODK Survey, the select_one_with_other prompt type is currently broken.

To use the 2.0 Synchronization protocol implemented with rev 126, you must use ODK Aggregate 1.4.5; it has changed considerably from the ODK Aggregate 1.4.4 implementation. The synchronization protocol has not been extensively tested, but we consider it to be complete; it should now handle arbitrary-size datasets, transmit UTF-8 (non-Latin) characters, and operate under non-App Engine deployments. This release attempts to minimize data transmissions by not transmitting files and data that have not changed. There are undoubtedly undiscovered bugs in this area of the code, as it is quite complex.

With respect to UTF-8 characters, there are some areas, such as UTF-8 value propagation into sub-forms, that are likely broken. We simply have not tested this area of the code well enough to confirm that it works.

Data tables with many columns will need to be split across multiple sub-forms (sub-tables) manually. The ODK 2.0 software no longer magically handles very wide surveys. The limitations of the underlying database are now exposed to the form designer. This means that if you develop on Google App Engine or PostgreSQL, and then later want to move to MySQL, you will likely encounter problems as the first two systems support database rows (submissions) of up to 1MB (3800 string fields), whereas MySQL only supports rows up to 65KB (240 string fields). The estimated number of data fields is greater if you are using integer, numeric or boolean values; each string field uses space equivalent to 60 of these other data types.

The ability to specify the size of a string field is not yet exposed. For now, string fields are limited to 250 characters or less.

Select-multiple fields store their selections all in one string field, currently limited to 250 characters. These fields will typically need to specify a larger field size once that capability becomes available.

Device Software

The user-visible functionality of ODK Survey and ODK Tables is unlikely to change going forward. We are only intending to add more administrative settings to hide portions of the user interface, and perhaps move the ODK 1.x publishing up into the server and off of the device entirely. Also, ODK Survey will likely support a custom framework form -- i.e., the one used in the Application Designer. This would be a configuration setting.

There will be several dramatic changes to the internals. First, the ODK 2.0 tool settings will be consolidated into one properties file that is synchronized across all devices and a set of device- and user-specific settings that are not. All of the ODK tool settings will be within this one file, with the settings for the various tools isolated through a naming convention. Second, the directories will be restructured to better isolate the configuration, data, and system files. Third, the various content providers and the internal web server that vend the form data and the webpages for ODK Survey and ODK Tables, and the database service that ODK Survey uses, will all be moved into the ODK Sync application, and that application will become a required component of the system.

Server Software

We will be adding publishers to external services. We may also move the ODK 1.x publisher functionality up into the server and out of ODK Survey.

We will also be adding the ability to control which devices and users get which subsets of data. This would allow different devices to see different slices of data (e.g., just the data they collect, or just the data appropriate for their needs). This is the future purpose of the filter_type and filter_value metadata columns in the data tables.

We also have plans to update the MySQL implementation and support Google CloudSQL with more relaxed column-length and survey-length limits.

Desktop Software

The App Designer now automates some of the tedious form design process. We will continue to revise this tool to require less manual editing, reloading and saving of files. We expect this to become a more integrated development environment with less need for access to the file browser. The ODK Scan form designer will also be added as a tab on the designer's main webpage.

Older Versions

Descriptions of older versions of the tools may be found here