HDX Testing

Owned by Michael Rans
Last updated: Jun 24, 2019 by Yumi Endo
7 min read


Internal testing draft plan below:

next step:

  • dev+design to align on which server to conduct a test
  • slimmed down version of file store on one of the non-production server
  • description of test for each feature - with help from CJ + nalae - with more randomness to test

A list of HDX features for periodic test:

*testing to be done on feature- . Please check with yumi/dan prior to testing to make sure feature- is in mirrored state as production. 

  1. Add and edit data 
    1. Add a private dataset with 3 resources under HDX Org
      1. (Login as HDX org admin account) Go to "Add Data" and create a private dataset (alternatively access "Add data" from HDX org page under the logo)
      2. Upload 3 resources 
      3. Fill out the mandatory fields (red starred marked fields)
      4. Publish the private dataset
      5. Logout from your admin/editor account. Try to find the private dataset under HDX org (it should not be visible to non-HDX org member). Login as either HDX org member or admin and try to find the private dataset you created(it should be visible)
    2. Edit private dataset
      1. Go to the private dataset page you created and click on "Edit"
      2. Change details of a resource
      3. Add 2 more resources and delete one of the old resources
      4. Switch the privacy setting from "Private" to "Public" and fill out all mandatory fields
      5. Publish
      6. Check errors on dataset page
    3. Repeat a and b for public datasets

    4. HDX Connect
      1. (Login with your HDX org admin account) Go to "Add Data" and add a (temp) HDX connect dataset. Fill out all required fields (*) and publish. 
      2.  (Logout from your HDX admin account) Find the HDX Connect dataset you created and(if you're not logged in - it should ask you to login. Login as HDX org editor or member) request access from "Data" view on main nav as well as from
      3. (HDX Connect) dataset page. Make 2 requests (1 you will approve, 2 you will decline as a maintainer).
      4. Check in mailhog - make sure you received the 2 requests with messages you had entered. (if the test was not performed on production - these emails are caught in mailhog and actually not sent. so check the mailhog inbox + click on link to request page) 
      5. Login again as a HDX admin + approve one of the requests (with attachment - use dummy xls or csv) and decline the other request. 
      6. Check in mailhog (one should be approved with attachment and the other should be declined).
      7. Finally, delete the HDX connect dataset. Check on "Data" view on main nav to confirm it's deleted.  
  2. Send "Contact the contributor" and "group message" (choose from: to everyone, all admin, all editors)
    1. (without logging in) Send a message to contributor from any dataset page. Do the same task while logged in (from any account) and make sure your email is pre-populated. Check in mailhog
    2. (Login as HDX org admin) Send a group message from any HDX dataset page. You can pick from all/all admins/all editors. Check in mailhog
  3. Send a question via "contact us" in FAQ page
    1. (without logging in) send a question from FAQ page + check in mailhog 
    2. (while logged in to one of the account - admin/editor/member - check that your email is pre-populated in the form) send a question + check in mailhog and check the email address of the sender.   
  4. New user registration  
    1. New signup 
      1. Sign up as a new user using a new email address
      2. Make sure you receive email to "verify your email address" 
      3. Click on the URL in the email to complete the registration (create user name, password, fill out your name etc)
      4. Follow different subjects - location, topic etc
      5. Request membership to request to join the existing organization or request to create a new organization 
      6. Check the membership request email in mailhog 
      7. Logout and Login again(using account you just created) - check that you're directed to Dashboard when you login 
    2. Approve new member as org admin
      1. (Login as HDX admin) Go to HDX org page > "Members" tab. (or if you're continuing test from a - you can access the member's tab from the email link in step vi)
      2. "Approve" the request and assign a role (admin/editor/member)

  5. Org admin tasks 
    1. Membership management - Invite a new user to join org
      1. (Login as HDX admin) Go to HDX org page. 
      2. Invite/add a new member from the 'members' tab. If the invited user is already signed up on HDX, the full name should appear as a dropdown option(by either entering the email address or full name) + should be immediately added as a chosen role(please refresh the page to check).
        If the invited user doesn't have account on HDX - she/he should receive an email to change password.
      3. Check in mailhog for 2 emails, 1 addressed to all members of the org that someone is added, 2 to the invited user to inform their account was created and need to update the password. 
    2. Membership management - delete a member in the organization
    3. Membership management - assign different role to a member in the organization
    4. Data maintainer - 
  6. Sysadmin tasks
    1. Make a dashboard /event page + edit existing dashboard/event page
    2. Add/Change carousel item on homepage
      1. Go to https://feature-data.humdata.org/ckan-admin/carousel/show (this URL may change in the coming weeks due to changes in security for sysadmin)
      2. Add a carousel item > publish > check on homepage for changes
      3. Delete a carousel item > publish > check on homepage 
      4. Edit a carousel item > publish > check on homepage 

    3. HDX Connect dashboard (only sysadmin can see)
  7. Data Check (A. validate a local file + remote resource on Tools + B. validate a hxl-tagged resource on HDX)
  8. Quick Charts (A. test making quick charts on Tools + B. After testing 2(add data with hxlated data), turn on  

Note: if you experienced any bug/issue, make a Jira ticket and tag Yumi/Dan.

Week of / Features testedAdd and edit data Send "Contact the contributor" and "group messageSend a question via "contact us" in FAQ page

New user registration  

Membership managementSysadmin tasks789101112
(example)
Week of 1/1/2019		x (no issue)x (Jira link)x (no issue)









Week of 6/10/2019



















































































Volunteer testing draft plan below

Dear Volunteer,

HDX runs on the open-source CKAN platform, and we are upgrading our version from 2.3 to 2.63 and adding a new feature, requestable datasets. As of today, the new version is ready on the feature server.

We’d like your support in testing this upgraded version on the feature server. The main goal is to confirm that the new version behaves the same way as the old one, and to identify any issues or problems.

We’ve provided more detailed instructions on this document. Sample actions you can test are:

  • Creating and modifying user profiles

  • Creating and modifying organisations

  • Creating and modifying datasets, including all metadata and resources and particularly requestable datasets

  • Adding gallery items to datasets

  • Testing the Python Library


A full list of testing scenarios is below. Feel free to choose any of them.

**Be sure to test on the feature server, not on the actual HDX site. Please use this template to record any issue you experience.**

You will receive an invitation to a Slack channel for ease of coordination. If you're not comfortable using Slack, please email questions to me at becky@humdata.org.

Thanks for your support!

Best wishes,

Becky

--

Testing scenarios/ideas

All testing to be done at this url: https://feature-data.humdata.org

User registration

  1. Sign up as a new user

  2. Request membership to a testing organization

  3. Check user dashboard

  4. Change your password and account details

  5. Check members page

  6. Check your role inside organization

  7. Contact the other members from organization by sending a group message

  8. Submit the Contact contributor form

Creation/edit datasets

  1. Create a private dataset with 3 resources

    1. create a private dataset

    2. upload 3 resources

    3. fill only the mandatory fields (red star marked fields)

    4. save dataset

  2. Edit  private dataset

    1. edit the previously created dataset

    2. change details of a resource

    3. add 2 more resources

    4. change from private to public dataset

    5. try to save

    6. check errors

    7. fill all the mandatory fields

    8. save dataset

  3. Edit private dataset

    1. edit the previously created dataset

    2. remove 2 resources

    3. add 1 new resources

    4. save dataset

  4. Edit private dataset

    1. edit the previous created dataset

    2. change to public dataset

    3. fill all the mandatory fields

    4. add 3 different type of resources

    5. save dataset

  5. Repeat all steps for public datasets

Requestable Datasets

Please click on the link to see tests for requestable datasets.

Locations

  1. Check locations landing page

  2. Check active location page

  3. Check inactive location page

QuickLinks (in the top navigation bar)

  1. Check each quicklink if it loads

  2. Check the data on each quicklink page

HXL Preview

If the data uses the HXL standard, then HDX can automatically create customizable graphs and key figures to help you highlight the most important aspects of your dataset. This is called a "HXL Preview." For a HXL Preview to be generated, your dataset needs:

A CSV with HXL tags as the first resource. HXL is easy! Check out the 30-second tutorial. The CSV can be stored on HDX or as a remote resource at another URL.

To be tagged with "HXL" on the dataset page. This is not referring to the HXL tags inside the CSV, but rather the metadata tags that you set for the whole dataset. This is a flag to HDX to try and process the CSV as a HXL-ated dataset.



  1. Upload a csv file that contains hxl tags and check if hxl preview appears.


Geo Preview

  1. Upload a zipped shapefile or geojson and check if the geopreview is generated (hint: for shapefiles, the file type must be set to "zipped shapefile" instead of the default "zip" for geopreview to be triggered)

Python HDX Library [for Python programmers]

  1. Test the HDX Ckan with the new site, trying as many different API calls and edge cases as you can think of.

HDX Homepage [desktop version]

  1. Go to homepage

  2. Check the search box in the top toolbar. When you start typing a word, do you see useful autocomplete suggestions?

  3. Repeat the same test with the search box in the centre of the page.

  4. Click on trending terms and check if the results appear reasonable

  5. Check each item from the carousel and click on each item to see if it opens the appropriate page

  6. Check page 2 of carousel

  7. Click on each item from footer and see if it opens the appropriate page

HDX Homepage [mobile version]

  1. Go to homepage

  2. Check if page displays in a readable way on a smaller screen (phone or tablet)

  3. Check top right menu and click on items

  4. Check the search box in the top toolbar. When you start typing a word, do you see useful autocomplete suggestions?

  5. Check the search box in the center of the page.

  6. Click on trending terms and check the results

  7. Check each item from carousel and click on each item. Carousel has to look different on mobile

  8. Click on each item from footer

Map Explorer [desktop embedded version]

  1. From home page click on the item of carousel (Explore)

  2. Add layers as you want

  3. Change choropleth to bubble

  4. Remove layers as you want

  5. Click on map and check filtering

  6. Click on chart and check filtering

  7. If user is logged in click on “save”. Follow the screens and check the links and information displayed

  8. If user is not logged in click on “login”


Map Explorer [desktop full page version]

  1. Url: https://test-data.hdx.rwlabs.org/mpx/#/

  2. Same actions as in above.

Map Explorer [mobile full version]

  1. From home page click on first item of carousel (Explore). On a mobile device you should be redirected to full page explore for mobile

  2. Add layers

  3. Remove layers

  4. Filter layers and charts

Data nerds!

  1. From the Map Explorer, click on “info” in any layer

  2. Download the data and compare to what you see in the map explorer

  3. Check that totals and values align with what is in the data