Application Designer - rev 126

The documentation on this page corresponds to rev 126 (February 2015) of the tools suite. See Older Versions for documentation on earlier releases of this suite.

This is an older version of our software.

The websites referenced in this documentation will generally have been updated to work with the newer software, and therefore may not function with this older release.

Where practical, we have provided zip files of the ApplicationDesigner environment used in creating those sites. See Survey and Tables Aux Files

Release Designation

ODK Application Designer rev 126 (February 2015) is designated Alpha software.

See Release Designations for the meaning of this designation.

Overview

The ODK Application Designer, when combined with Excel or OpenOffice for form design, the Chrome browser, and your favorite editor for template design, provides a development environment for the application designer.

In the context of the ODK 2.0 tools, application design consists of:

  • designing the forms used in data collection (by ODK Survey)
  • designing the HTML landing pages and screens used for navigating, curating, and visualizing that data on your Android device (within ODK Tables).
  • customizing the look-and-feel of both of these via customized images, logos, and CSS rules.

And, if you are only interested in data collection, or if you are only interested in data dissemination and visualization, you can focus your application design efforts on just that one tool (ODK Survey or ODK Tables, respectively). The two tools operate independently -- you are not required to deploy both tools to your Android device, or use both tools. Simply select the combination or individual tool that fits your needs.

The major goals of the ODK Application Designer are:

  1. Simplify the form-design process by providing a preview of your form with the same screen geometry as your target Android device. You no longer have to copy each iteration of your form onto your device to see how it will look or fit on a smaller screen.
  2. Simplify the design and testing of customized list- and detail- views in ODK Tables, and in the design of graphical representations of data within ODK Tables
  3. Simplify the customization of the look-and-feel of your forms through a simple visual theme editor / generator where your modifications can be immediately viewed and the resulting CSS styles or theme can be saved to a file for later incorporation into your application deployment.
  4. Simplify the conversion of the XLSX file into a form definition by providing a drag-and-drop conversion app running locally on your desktop.
  5. Simplify the transition to the use of ODK Survey as a replacement for ODK Collect by generating a stub XML form definition that can be uploaded to ODK Aggregate and used by ODK Survey to submit data into ODK Aggregate's XForms submission processing and publishing pipeline (the one used by ODK Collect). This enables organizations to leverage the 2.0 tools on their Android devices while still retaining their established backend procedures and processes. i.e., you can be early adopters of these new Android device technologies while retaining the stability and predictability of the well-established, proven, ODK Aggregate XForms processing pipeline.

The general layout of this document is:

 

Overview

This section presents a brief overview of the features of the ODK Application Designer. To fully utilize the system, you must understand the layout of the files on the Android device and the specific steps to create them.

These details, and how to set up the ODK Application Designer tool, will be covered in the subsequent sections.

The ODK Application Designer is accessed through a Chrome browser. Once launched, it opens Chrome to display:

Initial Screen

This screen has 5 tabs:

  • Preview (shown) - used to preview ODK Survey forms and ODK Tables list-, detail- and graph- views (future). Displays these within a user-selected device geometry.
  • Customize - a visual style and visual theme editor. This editor immediately shows the effects of changes to specific settings in your CSS file.
  • XLSX Converter - converts the XLSX description of a form into a formDef.json representation used by ODK Survey.
  • Legacy XML - converts the formDef.json file into a stub XForms XML file that can be uploaded to ODK Aggregate to support publishing data into the XForms-based submissions pathways used by ODK Collect.
  • File Browser - enables browsing of the directory structure that will exist on your Android device so that you can access or view other files (currently necessary for accessing the ODK Tables list- and detail- views).

 

Preview

The Preview tab (shown above) has several controls:

  • Launch Page -- what webpage to launch in the window below.
  • Purge Database -- during development, if you are adding new fields or changing their data types, you will need to purge the database so that the database structure can be re-generated with the proper fields and data types.
  • Device Dimensions -- what dimensions to make the window below.
  • Back Button -- this is generally only useful when viewing ODK Tables web pages. It may cause unpredictable results when used while displaying ODK Survey forms.

The Launch Page selections are expected to grow with later updates. Presently, you can select among:

  1. ODK Survey Framework Page - opens the formDef.json in the Android device's application frameworks folder (e.g., either /sdcard/opendatakit/survey/frameworks or /sdcard/opendatakit/tables/frameworks). The contents of this form are defined by the framework.xlsx file in that same directory.
  2. ODK Tables Graph Page - opens the ODK Tables graph creation page against the Myna Birds dataset.
  3. ODK Survey Step-Through Test - opens and advances through the Example Form. This is a simple example of a test of form navigation.
  4. ODK Tables API Test Page - runs a series of tests to verify that the expected values are returned from the simulated ODK Tables java application interface into the HTML and Javascript page.

For example, if you click on the household_new test form, and click the Follow Link button on the next screen, the Household Survey form is launched, yielding this screen:
After following link to Household Survey

You can navigate through forms, enter and exit sub-forms, and save results just like on your Android device.

The development environment does not allow you to submit data to a server (future work).

Customize

The Customize tab contains the CSS style and theme generator:
CSS Style and Theme Generator
Using this tool, you can change background colors, fonts, and other settings affecting the appearance of a form. The changes are reflected immediately in the form shown to the left of the toolbar.

See ODK ThemeGenerator documentation for more information about this tool.

XLSX Converter

The XLSX Converter tab contains the conversion tool that transforms XLSX files produced by Excel or OpenOffice into the formDef.json file used by ODK Survey:
XLSXConverter

See ODK XLSXConverter documentation for more information about this tool.

Legacy XML

The Legacy XML tab contains the conversion tool that transforms a formDef.json file into a Stub Form Definition that can be uploaded into ODK Aggregate to enable ODK Survey to send submissions into ODK Aggregate's XForms XML submissions path (the one used by ODK Collect):
Legacy XML generator

See ODK Legacy XML documentation for more information about this tool.

File Browser

The File Browser tab provides a view into what will become the application's directory on the phone.
File Browser

The default application directory on the Android device for ODK Survey is /sdcard/opendatakit/survey/. You can set up ODK Survey shortcuts (widgets) to start ODK Survey in an alternative application directory within /sdcard/opendatakit/. ODK Tables currently can only launch into its default application directory (/sdcard/opendatakit/tables/); this is a temporary limitation.

This tab is most useful when developing ODK Tables applications and wanting to test the HTML and Javascript of those pages.

For example, to view the Tea Time in Benin demo's tea houses list, navigate and select:

/app/tables/Tea_houses/html/Tea_houses_list.html

This yields:
Tea_houses_list.html

And from here, you can navigate through the Tea Houses screens. e.g., if you click on the topmost entry, you are presented with:
Tea House Detail

In future updates, you will be able to open these files from within the Preview screen so that you can view them with the scale and geometry of your target Android device.

 

Getting Started

You must install the following software on your computer:

  • Chrome - Google's Chrome browser. Download and install it from Chrome.
  • NodeJS- a framework for easily building fast, scalable applications. Download and install it from NodeJS
  • Grunt - a task-based scripting environment (installation is described below).
  • Android SDK - the software development kit for Android devices (installation is described below).
  • ODK Application Designer v2.0.zip - a zip containing this tool (installation is described below).

It is tricky to foresee all the issues that can crop up on many different machines and setups. If something in this process does not go as expected, please tell us. Others are likely encountering the same problem, and we can only make these instructions more complete if people tell us when the instructions are incomplete!

NodeJS

Follow the instructions here to install NodeJS.

For Mac/Unix:

After installing NodeJS, open a terminal (which you can do by clicking the spotlight in the top right corner of the screen, typing terminal, and clicking the program named "Terminal") and type npm --version. If a number is not displayed, but you instead receive a message that the command npm cannot be found, you will have to perform some additional configuration.

As of this writing, by default NodeJS installs its commands into /usr/local/bin/. In the terminal, type ls /usr/local/bin/npm. If this command outputs something like /usr/local/bin/npm, but you are still unable to run npm --version, try running /usr/local/bin/npm --version. If this is successful, then npm is successfully installed, and you will just have to add /usr/local/bin/ to your system PATH variable (see below).

If the command ls /usr/local/bin/npm outputs a message telling you permission is denied, then you will have to change the ownership of the /usr/local/ and /usr/local/bin/ directories. On Mac, follow the instructions here to take ownership of these directories, or to at least give yourself read permission. On other Unix systems, use the chown command or the user-interface appropriate to your distribution to do so.

Grunt

After installing NodeJS, install Grunt by doing the following:

    1. On Windows, open a cmd window (go to Start Menu, search for cmd and open it); on MacOSX, open a Terminal window.
    2. Within this window, type
npm install -g grunt-cli

Note: these installation steps are copied from here. If grunt is successfully installed, the following command:

grunt --version

Should display the installed version of grunt. e.g., grunt-cli v0.1.13

Android SDK

To install the Android SDK:

  1. Browse to here.
  2. Scroll down on this page and click to expand the GET THE SDK FOR AN EXISTING IDE section.
  3. Within that section, select Download the SDK Tools...
  4. Accept the license agreement
  5. Wait for the install of the SDK Tools to complete.
  6. Run the SDK Manager
    • On Windows, it is available in the Start Menu under Android SDK Tools
    • On Mac/Unix, open the SDK folder you downloaded above. In the bin/ or tools/ directories (on some versions it is in both places--it doesn't matter which you use), double click the file called android.
  7. Select the latest versions of the following packages (by checking their checkboxes):
    • Android SDK Tools
    • Android Platform-tools
    • Android Build-tools
  8. Click the Install 3 packages button in the lower right corner of the screen.
  9. A licensing pop-up dialog will appear. Accept the license agreement(s) by selecting the 'Accept License' option. If there are multiple licenses, you may need to select each license in the 'Packages' window on this dialog and check this 'Accept License' option for each of them before the 'Install' button will become enabled.
  10. Click the 'Install' button on that dialog to begin the installation process.

Among many other things, this will install the Android Debug Bridge software on your computer. This tool enables the scripted pushing of files and APKs down to your Android device. See adb (Android Debug Bridge) for a listing of its capabilities.

Next, open a cmd window (open the Start menu, type cmd in the search box, select and open it), and type:

adb version

If there is an error complaining about Java not being installed, you will need to close this cmd window and download and install Java. After installing, Java, open a new cmd window and type this command again.

If this displays a version string, then your installation is complete; skip to the next section.

Otherwise, if adb is not found, then you need to add it to the PATH variable of your system.

For Windows:

    1. open a Windows File Explorer and navigate to the location of your Android SDK. This will typically be at one of: C:\\Users\\your_username\\android-sdks or C:\\Program Files\\Android\\android-sdk or C:\\Program Files (x86)\\Android\\android-sdk
    2. navigate into the platform-tools folder.
    3. Click in the file path at the top of the File Explorer window. The path will become a selected text string. Copy it into your copy-and-paste buffer.
    4. open the Start menu,
    5. right-click on Computer,
    6. choose Properties. The System Control Panel screen opens.
    7. select Advanced system setting on the left sidebar. The System Properties dialog opens.
    8. click on the Environment Variables... button at the bottom of the screen. The Environment Variables dialog opens.
    9. select the Path variable in the bottom System variables scroll window.
    10. click Edit...
    11. click into the Variable value text box
    12. press your End key to move to the very end of this extremely long string.
    13. enter ; and paste the platform-tools directory path after that semicolon.
    14. click on OK and exit all of the windows.
    15. verify that you have made the change by closing all cmd windows and open a new one (so it picks up the change), and type
adb version

You should now see the version of the adb tool. e.g., Android Debug Bridge version 1.0.31.

For Mac/Unix:

The PATH variable is nothing more than a default list of places the system looks for commands. Open a terminal.

Type echo ${PATH}, and you will see a colon-separated list of folders on your computer. (echo means just print whatever comes next, and the ${ } means that the system will treat PATH as a variable, not a program. You don't need to know this to follow these instructions, but knowledge is power.) For example, you might see something like this:

$ echo $PATH
/usr/local/bin:/usr/local/sbin:/usr/bin:/bin

This means that when you type adb --version, the system will look for the command called adb in the directories /usr/local/bin/, /usr/local/sbin/, /usr/bin/, and /bin/.

Note the location where you downloaded the Android SDK. It should contain a folder called platform-tools, which itself contains the program adb. If this was in the folder /Users/someuser/Desktop/android-sdk/ you should be able to run /Users/someuser/Desktop/android-sdk/platform-tools/adb --version. This works because we're telling the computer exactly where the program adb exists. By putting the platform-tools directory on the system's PATH variable, we will be able to just type adb and have the system find it in the /Users/someuser/Desktop/android-sdk/platform-tools/ directory.

Sadly, this process is more involved on Mac/Unix than on Windows. Use a text editor (not Word, but something like TextEdit), select the option to open a file, and browse to your home directory. You can find your home directory by typing echo ~ in a terminal. (~ is a shortcut for the home directory.) Macs use a hidden file called .bash_profile in the home directory to set variables like PATH. Other unix systems use files like .bashrc You might have to check the specifics for your distribution to know which you should use. Open the appropriate file. If the file does not already exist, create a new file that will be saved with the appropriate name in your home directory.

We want to add the location of the adb tool to your PATH while preserving the existing PATH information. Assuming that your adb program is in the /Users/someuser/Desktop/android-sdk/platform-tools/ directory, you would add the following command to the end of the .bash_profile file:

export PATH=${PATH}:/Users/someuser/Desktop/android-sdk/platform-tools

Save the file, close the terminal window, open a new terminal window, and type echo $PATH. You should see your old path with the new directory you added above, and you should now be able to run adb --version.

If you are going to be heavily customizing the look-and-feel of the application with a lot of external JavaScript libraries, you might also choose to install bower.

ODK Application Designer v2.0.zip

Download ODK Application Designer v2.0.zip from the Survey and Tables portion of our downloads page, here; this documentation is for version 2.0_alpha_2_rev_126.

Unzip this file and move it to somewhere other than your Downloads directory; e.g., to your Documents folder.

You will be opening a 'cmd' window and changing your current directory ('cd'ing) into this directory every time you use this tool. It is therefore useful to create a shortcut that opens a 'cmd' window directly into this directory.

On Windows,

  1. Open a file browser and navigate to the unzipped directory containing a number of files and directories, including a Gruntfile.js.
  2. Click into the top location bar that displays the nested list of folders to this folder.
  3. Copy this path to the cut-and-paste buffer
  4. Now, move down to the list of files, right-click
  5. Select 'New...', 'Shortcut'
  6. Type 'cmd' for the location of the item.
  7. Click Next, and then Finish.
  8. Select this newly-created 'cmd.exe' shortcut and right-click
  9. Select 'Properties'
  10. Click on the 'Start in' text box, delete its contents, and paste the path to this folder.
  11. Click 'OK' to accept the change.
  12. Double-click the 'cmd.exe' shortcut to open a 'cmd' window.
  13. Confirm that it opens in the intended directory (you should see the full path to that directory displayed to the left of the blinking cursor).

On MacOSX, Terminal will open a new terminal window if you drag a folder (or pathname) onto the Terminal application icon, and you can also drag a folder to the tab bar of an existing window to create a new tab in that folder.

You have now completed the installation of the ODK Application Designer software.

 

Launching the Application Designer

The ODK Application Designer is both (1) a workspace containing the cohesive interacting set of forms and files you have created and (2) a set of tools used for that development.

We recommend unzipping and creating a new ODK Application Designer directory for each new set of ODK Survey forms and ODK Tables files that are not intended to be deployed as a cohesive unit. With the eventual availability of the ODK Application Packager, you will then be able to package each set of forms and files into a separately installable APK that you control. If you need to have several of these sets of forms and files co-resident on the same Android device, you would simply install the required set of your APKs on that device. The underlying ODK 2.0 tools will then keep each of these sets of forms and files isolated from each other.

To launch the application designer, open the 'cmd' shortcut or Terminal window onto the directory containing Gruntfile.js and type:

grunt

This should automatically open Chrome and display the Preview tab. When you need to stop the server, return to the command or terminal window where you typed grunt and press ctrl+c (that is the control and the letter c key at the same time). This will stop the process.

If you have Parallels or other virtualization software running, it might try to open Chrome in this system by default. If so, you should still be able to navigate to the url below.

If the Chrome browser does not open, try opening it yourself and browsing to http://localhost:8000/index.html.

If the page never times-out, but never loads (it remains blank or constantly spinning), then stop grunt and try this command instead:

grunt --verbose connect:livereload:keepalive

This will start grunt, but disable the file-change detection mechanisms that automatically reload an HTML page when it or any javascript file it uses has been modified. If you need to run the command in this manner, please add a comment to this issue so that we can guauge how widespread this problem is. Others have reported that uninstalling npm and node, and then re-installing them may correct the issue. If you try that and it works, please also add a comment to the above issue.

 

Directory Structure

There are many folders and files within the Application Designer directory. Fortunately, the only ones that are of interest for a non-software-developer are:

  • app/ - folder containing everything that will be pushed to the Android device.
  • Gruntfile.js - contains the definitions of tasks that push files to the Android device, launch the Chrome browser, and pull data and log files off the Android device.

Initially, you will only be concerned with the contents of your app/ directory -- the set of files that are placed on the Android device. As your sophistication grows, you may want to define your own grunt tasks to automate repetitive steps in your deployment and device management processes. Adding or modifying tasks is beyond the scope of this document; please refer to the grunt website (see Getting Started for the link to that site).

For completeness, here is the full list of the files and sub-folder in this directory. Again, you generally do not need to be concerned with the contents or specifics of any of these:

  • app/ - folder containing everything that will be pushed to the Android device.
  • convertToLegacy/ - contains the HTML and Javascript for the ODK Legacy XML stub XForms XML generation tool (accessed via the Legacy XML tab).
  • devEnv/ - contains the HTML for the 5 tabs of the Application Designer.
  • grunttemplates/ - contains template files used by grunt tasks.
  • node_modules/ - contains additional software installed by npm, such as external tools used by grunt.
  • test/ - contains tests of the computer-based simulated device environment.
  • themeGenerator/ - contains the HTML and Javascript for the ODK ThemeGenerator CSS style and theme customization tool (accessed via the Customize tab).
  • xlsxconverter/ - contains the HTML and Javascript for the ODK XLSXConverter tool that converts XLSX form definitions into formDef.json files (accessed via the XLSXConverter tab).
  • .bowerrc - JSON configuration for the 'bower' tool.
  • .editorconfig - when your text editors are configured to use it, enables consistent formatting to files across all contributors to your application design. See EditorConfig.
  • .hgignore - source code management configuration.
  • .hgtags - source code management configuration.
  • .jshintrc - configuration for JSHint - a program that flags suspicious usage in programs written in JavaScript.
  • bower.json - used to control library management through bower. By default, the .bowerrc file has been configured to install these libraries in app/framework/libs so that you have access to them when your app is pushed to the phone.
  • deleteDefAndProp.sh - MacOSX shell script to traverse the relevant parts of the app/ directory and delete the definition.csv and properties.csv files.
  • Gruntfile.js - contains the definitions of tasks that push files to the Android device, launch the Chrome browser, and pull data and log files off the Android device.
  • index.html - the main HTML for the ODK Application Designer web page.
  • macGenConverter.js - MacOSX command-line wrapper for the XLSXConverter tool (converts a single XLSX file piped into stdin into a formDef.json on stdOut).
  • macGenFormDef.sh - MacOSX shell script to traverse relevant parts of the app/ directory and generate formDef.json files from XLSX files.
  • package.json - configuration information for npm.
  • README - description linking back to this document.

 

The app/ Folder

Everything in this folder will be pushed to the Android device. The directory looks as follows:

  • assets
  • framework
  • output
  • tables

The app/assets/ folder

Place any files that might not be associated with a specific form or table in this directory.

This folder contains:

  • css/ - contains the common CSS files for ODK Tables detail, list and home screens, and for app forms in ODK Survey (odk_survey.css).
  • csv/ - contains the data files to be initially read and loaded into the ODK Survey and Tables databases. NOTE: this functionality is currently not operational within ODK Survey.
  • img/ - contains the images used throughout the application.
  • js - contains Javascript used by the ODK Tables custom home screen and/or the ODK Survey custom forms list. NOTE: this functionality is currently not operational within ODK Survey.
  • config.properties - contains the initialization directives for which data (csv) files should be loaded at initial start-up of the ODK Tables or ODK Survey application NOTE: this functionality is currently not operational within ODK Survey.
  • index.html - the HTML for the ODK Tables custom home screen, if it is enabled in the ODK Tables configuration settings.

Eventually, this folder will also contain the formDef.json of the ODK Survey custom forms list. This file is currently located under the framework directory and is currently only accessed within the Application Designer environment.

It is also expected that the translations of the framework's text strings (e.g., translations.js) will also reside within this directory.

The app/framework/ Folder

This folder contains the files that the ODK Survey and ODK Tables depend upon and which are expected to be changed only when different versions of the ODK Survey and ODK Tables APKs are released.

The general structure is:

  • graph/ - contains Javascript needed to implement the specific types of graphical displays supported by ODK Tables and ODK Survey. ODK Survey currently does not have any graphical prompt types (future work).
  • libs/ - contains 3rd party Javascript libraries used by ODK Tables and ODK Survey.
  • survey/ - contains Javascript used by ODK Survey to render forms.
  • tables/ - contains Javascript used by ODK Tables to render the custom home screen, list, detail, and graphical views created by the Application Designer.
  • formDef.json - the custom ODK Survey forms list screen definition. Will be moving to the assets/ folder.
  • framework.xlsx - the form definition used to create the custom ODK Survey forms list screen (above).
  • index.html - the generic HTML for all ODK Survey forms.
  • manifest.appcache - possibly needed for improved offline performance (currently disabled).
  • optionspane.html - the ODK Tables graph-creation page.
  • surveyTest.html - the simple walk-through test page for ODK Survey.
  • translations.js - the translations (i18n) for ODK Survey. Will be moving to the assets/ folder.

The goal is that only the ODK core team should have to modify things in this folder.

The app/output Folder

The ODK Application Designer provides various grunt tasks to pull files off the Android device. These files include json objects for debugging, exported csvs, and the database itself. The grunt tasks store these files here.

The app/tables Folder

This folder has a predefined directory structure, but the content is entirely dependent upon the needs of your application.

The zip file for the ODK Application Designer populates this with all the sub-folders used by each of the ODK Tables and the ODK Survey demonstration zip files. Ultimately, when you have completed your application design, this folder will contain none of these original folders but would instead contain only the folders which you have created.

Unlike ODK Collect, which stores each submission in a separate file, ODK Survey and ODK Tables store their combined collected submission data in data tables (one row per submission).

ODK Tables can display the contents of a table through one or more custom list views; it can display individual submissions through one or more custom detail views. ODK Tables will also support custom graphical views. All of these custom views are defined here (see below for details).

ODK Survey, unlike ODK Collect, has the additional flexibility of supporting multiple forms to create, access and update data within a single common data table. This enables creating multi-stage workflows such as initial screenings and follow-ups, or registrations and status-updates (submission data can be editable, or not, based upon the form used at that workflow stage).

To accommodate these various capabilities, the tables directory is structured such that individual data tables each have their own directory within the tables directory. The table_id of the table is the name of this sub-directory. Table ids are specified at the time a table is created within ODK Tables, or by the formDef.json that defines an ODK Survey form. If not explicitly specified, ODK Survey will assume the form id and table id are the same.

The app/tables/table_id/ Folder

A canonical table_id sub-folder contains:

  • forms/ - contains directories for each ODK Survey form that manipulates this table. The names of these sub-directories are the form ids of those forms. Within each sub-directory, there is a formDef.json defining that form. If the form has form-specific images or media files, custom CSS, layouts, or prompt types, those files should reside within the form's sub-directory (nested sub-folders are permitted). If ODK Tables uses a custom ODK Collect form definition it is kept in the app/tables/table_id/forms/collect/ directory. This directory will be moved up to be underneath the table id sub-folder.
  • html/ the custom HTML files for the ODK Tables list and details views of the table's contents.
  • instances/ - contains up to one folder for each submission (instance) in the table. These folders are created only if the submission has an attached file (e.g., image, audio recording, arbitrary data file). The name of the sub-folder is derived from the unique instance id of the submission.
  • js/ the JavaScript files needed for the custom ODK Tables HTML list and detail views (found in the html/ directory).

 

Common Tasks

There are several common tasks that are facilitated by the ODK Application Designer. These are:

Each of these will now be discussed.

ODK Survey: Designing a Form

This section assumes that all of the necessary software has been installed and verified. If not, complete the instructions in Getting Started.

When creating a new form, the appropriate directory structure must be created. Once this directory structure is in place, an xlsx form can be created. From this xlsx form, a formDef.json file will be generated using the XLSXConverter. This formDef.json, in the appropriate directory, is what the system will use to create the survey form.

Creating a Canonical Directory Structure for a New Form

New forms must be placed under the app/tables directory as described in the app/tables folder section. Given a form with the name formId, it will have a tableId of the same name unless you explicitly specify otherwise. The directory structure that should be created is app/tables/tableId/forms/formId (where, under many circumstances, the value for tableId will be the same as the value for formId). To get started, open a cmd window within your ApplicationDesigner folder (click the cmd shortcut you created earlier), and type:

grunt addtable:tableId

This will create the required directory structure for an individual table, including the forms directory. Navigate into that, and create your formId directory. Within that directory, ODK Survey expects to find the formDef.json that defines the form. We recommend placing the xlsx file used to generate that formDef.json in this folder. Any custom screen or prompt templates or other media related to the form should be also placed in this directory (or in a sub-directory). For example, if you wanted to create a new form named testForm.xlsx, the directory structure that would be have to created for this form would be app/tables/testForm/forms/testForm.

Creating an ODK Survey

With the proper directory structure in place, you can now create your survey form. The XLSXConverter documentation extensively details the worksheets that will need to be created within your xlsx file to create a survey. Also, you can use the File Browser window of the Application Designer to navigate to example xlsx files under the /app/tables directory; it will likely be easier to start with one of the existing example forms and modify it. The key modification would be on the settings page -- changing the values for form_id and table_id (if present).

Generating a formDef.json File with XLSXConverter

Once you have a saved your survey XLSX file, you can use the XLSXConverter to create a formDef.json. The output from the XLSXConverter should be saved into the formDef.json file in that directory. e.g., form the 'testForm' form_id, you would save the result of the XLSXConverter in the file app/tables/testForm/forms/testForm/formDef.json. You can use the Save button to do this, or, we typically keep an editor open with the contents of the formDef.json file and copy-and-paste the XSLXConverter output into the editor, and save the changes. On Windows, this saves several awkward steps of deleting and renaming files.

You will notice that the form still does not appear in the list of forms found under the Preview tab of the Application Designer. In order for the form to appear, the framework.xlsx file in the app/framework directory must be modified, and the formDef.json file in that same directory updated.

The framework.xlsx file is just another form definition, but it generally has no persisted data. In this case, it just presents a list of forms and allows you to open them. Eventually, ODK Survey will offer a configuration option to use this form as its starting screen, rather than presenting the list of all available forms to the user.

The modifications to the framework.xlsx are as follows. Assuming you have created a testForm.xlsx, the appropriate directory structures for testForm.xlsx, and then properly generated and saved the formDef.json, the following lines would need to be added into the framework.xlsx survey worksheet.

branch_label url clause condition type values_list display.text display.hint
testForm
''?' + opendatakit.getHashString('../tables/testForm/forms/testForm/',null) external_link Open form
exit section

The following changes will also need to be made to the framework.xlsx choices worksheet

choice_list_name data_value display.text
test_forms testForm testForm

The changes to the choices sheet adds the testForm form as one of the choices that is shown in the user_branch prompt (a user-directed branching prompt type). The changes on the survey sheet add a branch label ('testForm') that matches the 'data_value' from the choices sheet (this branch label will be jumped to if the user selects the testForm selection on the user_branch screen). The new branch label then renders an external_link prompt type that has the necessary arguments to open the testForm.

Once you have made these changes and used XLSXConverter on the framework.xlsx file to update the app/framework/formDef.json file, you should see your new form show up in the Preview tab of the Application Designer. Clicking on that should open your form.

Debugging your Survey

The XLSXConverter should report most problems with your survey. If your survey generates a formDef.json without an error, but the form is not being rendered correctly, first try stopping the grunt command (on Windows, Control-C should produce a prompt asking to confirm whether to stop or not), and re-running it. Grunt can sometimes get overwhelmed with changes and stop working. After restarting, test your form.

If there are other problems, the contents of the Javascript Console will be helpful to the ODK core team for debugging. Open the Javascript Console by clicking the icon with the three bars in the top right, select 'Tools' and then select Javascript Console. You can click-and-drag to select all of the debugging output, then copy it, save it to a file, and send the file as an attachment to the opendatakit@ list.

Moving the New ODK Survey Form to a Device

NOTE: You must have USB debugging enabled on your device in order to perform this step. See here for instructions.

In order to push these changes to an Android device, you must first have ODK Survey installed on your device. Then:

  1. Force Stop your ODK Survey application by going into the device's Settings / Applications page, selecting ODK Survey, and clicking Force Stop. ODK Survey does not behave well when its files are changed out from underneath it.
  2. Connect the device to your computer via a USB cable
  3. Open a cmd/terminal window within the ApplicationDesigner directory (the one containing Gruntfile.js), as described in the Directory Structure section.
  4. Type: grunt adbpush-survey

This will copy all of the appropriate files for survey onto your device. You should then be able to launch ODK Survey, and it will display your form in its list of forms. Click the form to open it.

More grunt commands can be found in the Both: Pushing updates to the Device section, below.

ODK Survey: Altering the Appearance of a Form

The appearance of ODK Survey forms can be easily customized with the Theme Generator. You can use the Theme Generator by clicking on the Customize tab of the Application Designer. The Theme Generator creates a css file that can be used with your survey form to customize the colors and fonts.

Instructions for creating a css file for your survey can be found in the Theme Generator documentation.

ODK Survey: Setting up a legacy XForms XML Submission Path

ODK survey forms can be uploaded to Aggregate via the legacy XML XForms submission path (the one used by ODK Collect). Instructions on how to do this can be in the Legacy XML documentation

ODK Tables: Designing a Custom View

One of the most powerful aspects of ODK Tables is its ability to run HTML and
JavaScript pages as the skin of the app. Through a JavaScript API presented to these files, you can query the database and control the app.

Writing an app using html/js yields a lot of power. However, it can lead to a complicated design cycle.

The html/js files you write rely on the JavaScript API implemented within the ODK Tables APK to vend data-table values into your HTML pages, where they can be displayed as a list of items, as a detail view of a single item, or graphed in any number of ways. This Javascript API, since it is implemented in the APK, makes it difficult to debug your custom views off the phone. The solution is to add functionality to ODK Tables so that it can export a subset of your data-table contents plus additional configuration information that can then be used on your computer to provide an approximation of the device's capabilities within the Chrome-based ODK ApplicationDesigner.

 

Understanding the Web File

There are several pieces of boilerplate you have to include in your own code in order to debug the files in Chrome.

In the ODK ApplicationDesigner, use the file browser to open the tables/Tea_houses/html/Tea_houses_list.html file for the list view of the "Tea_houses" table. Right-click, and 'View Frame Source'. This shows the contents of that file. The important part to note is the following lines in the <head>:

<script type="text/javascript" src="../../../framework/libs/jquery.1.10.2.js"></script>
<script type="text/javascript" src="../../../framework/tables/js/control.js"></script>
<script type="text/javascript" src="../../../framework/tables/js/data.js"></script>

In the first line you are just making the jquery object available to your code; JQuery is a very powerful, very commonly used, set of functions for accessing and performing actions within a webpage. In the second two lines you are adding the control and data objects if they are not already provided by the browser environment. When running on the device, the ODK Tables APK will provide these, and the contents of these files will be ignored. When running on your computer, these files provide the approximate functionality of the APK, allowing you to create and debug your scripts.

Next look at the tables/Tea_houses/js/Tea_houses_list.js javascript file for the list view of the "Tea_houses" table. The important part to note is the following:

if (JSON.parse(control.getPlatformInfo()).container === 'Chrome') {
    console.log('Welcome to Tables debugging in Chrome!');
    $.ajax({
        url: control.getFileAsUrl('output/debug/Tea_houses_data.json'),
        async: false,  // do it first
        success: function(dataObj) {
            window.data.setBackingObject(dataObj);
        }
    });
}

This code does another check to verify that you are running inside the browser, and if so, initializes the 'data' object to use data-table content (Tea_houses_data.json) previously output by the ODK Tables APK.

 

Writing Your Own Web Files

To write your own file, first decide on the tableid for your table and instantiate a directory using the grunt command:

grunt addtable:tableid

This grunt task creates the needed directory structures and also constructs the html and javascript files with the necessary features for working within the Chrome development environment.

These files need content from your data-table in order enable you to begin creating your custom screens. The steps to accomplish that are outlined in the next section.

Once the content is available on your computer, you should be able to use the File Browser within the ODK ApplicationDesigner to open the custom views created via the grunt addtable:tableid command, earlier, and begin accessing the 'data' and 'control' objects with the JavaScript API to access columns and row values within the data.

Each time you save your changes to the html or javascript files, the ODK ApplicationDesigner environment will reload the page, enabling you to immediately see the effects of your changes. If this does not seem to be happening, terminate the grunt command, close your Chrome browser and re-run grunt (Grunt sometimes gets confused and this will reset it).

As with ODK Survey, you can use the Javascript Console to look for and fix errors in your HTML/Javascript. It will be beyond the abilities of the ODK Core Team to assist everyone in debugging their own custom javascript. Please search the web for answers. Keep in mind that the debug objects only emit a subset of the data in your ODK Tables database.

ODK Tables: Obtaining Sample Data Values for Testing

Obtaining sample data values for the design of your custom views begins with creating and populating your data-tables on your Android device. The easiest way to do this is to:

  • define an ODK Survey form whose formId matches its tableId.
  • click the Save To Filesystem button on the XLSXConverter. This will generate and save a definition.csv and properties.csv file under the app/tables/tableId directory.
  • push everything to the device as a tables application.
  • launch ODK Tables, and you should see this new table.
  • Configure ODK Tables to launch the ODK Survey form when editing or adding a new row.
  • Choose to add a row (click the '+' button) and use ODK Survey to generate your sample data.

Once you have created sample data on your device and it is visible within ODK Tables, start ODK Tables, click the gear icon in the top right of the screen and select Write Debug Objects to write a subset of your data-table values to 'debug object' files on your device.

Pull these files to your computer via:

grunt adbpull-debug

This will overwrite the debug objects initially provided with the ApplicationDesigner with the ones specific to your application. As a result, the Tea Time demo's custom views will no longer work within this ApplicationDesigner. If you want to refer to them, you may want to maintain an original version of the ApplicationDesigner and a working version for your application.

You are now done. You should be able to begin accessing this data via the html of the custom views you created earlier.

Both: Pushing updates to the Device

NOTE: You must have USB debugging enabled on your device in order to perform this step. See here for instructions.

There are several times during app development where you will need to push and pull files to and from the phone. You will have to open Collect and Tables once on the device before these commands succeed.

Push:

  1. The entire app directory
  2. The collect forms

Pull:

  1. The debug objects
  2. The database
  3. Exported csvs (if they were output to output/csv/

Grunt tasks have been written in Gruntfile.js that perform these operations for you.

These commands can be run anywhere within the repository.

Push:

  1. adbpush-tables-app. Pushes everything under the app directory to the device.
  2. adbpush-tables-demo-alpha2. Push only those files for the Tables alpha 2 demo to the device. This is how we generated the zip directory on the Downloads page. This task will not be present in future releases.
  3. adbpush-survey. Pushes everything required to run Survey to the device, but does not push the code required to run Tables.
  4. adbpush-survey-demo-beta2. Pushes the files required to generate the zip directory on the Downloads page.
  5. adbpush-tables. Pushes only the tables file to the device. This differs from adbpush-tables-app in that it does not push any files required by Survey. This is sufficient if your apps do not run on Survey. It can also serve to speed up the development process of a Tables app by not pushing the Survey files, which won't change between iterations.
  6. grunt adbpush-collect. Push any xml files under tables/tableId/forms/collect/ to /sdcard/odk/forms/, where tableId is any tableId present in the directory.

Pull

  1. grunt adbpull-debug
  2. grunt adbpull-db
  3. grunt adbpull-csv

You can perform both the adbpush-tables-app and adbpush-collect tasks by running grunt adbpush. Running grunt adbpull will perform all the pull tasks.

There are a number of additional grunt tasks available. Assuming you have installed grunt and node, you can view the available tasks by running grunt --help anywhere in the repo.

 

Testing

There are several automated tests that can be run on the device or on the computer

 

On the Phone

Start ODK Tables and import all the csv files in app/framework/test/csv (this is done automatically if you are using the demo files zip). They should be named:

  • Tea houses
  • Tea inventory
  • Tea types
  • Tea houses editable

The spaces will be replaced by underscores and the table ids will be correct.

In the Tea houses table, change the type of the Customers column to Integer.

Open Tea houses and load a list view with the file app/framework/test/html/javascriptTests.html. This will test the JavaScript API.

Open Tea houses editable and load a list view with the file app/framework/test/html/inputTests.html. Many tests will fail. Note the instructions in the link, click it, and finalize the form. Tests should begin passing. Note that you must have opened Survey and Collect in such a way to make the custom forms available. Further, note that these tests are currently failing.

 

On the Computer

There are also tests for the control.json and data.json objects in the browser. Run the webserver and navigate to the page test/index.html. This tests that the framework objects retrieve information from the debug objects correctly and that the API exposed to both the browser and phone web files remains consistent.