In this article, we will give you an overview of how to write tests for R and Python projects, both Guided and Unguided variants.

Writing Tests in R

Overview of testing in R

Testing for R projects is done using Hadley Wickham's testthat package. After loading in testthat and IRkernel.testthat in the first @tests cell, tests can be included without any additional code in the remaining @tests cells.

Examples of testing in R

Below are some examples of tests from our A Visual History of Nobel Prize Winners (R) Guided Project.

Test that a package has been loaded successfully:

run_tests({
test_that("Test that tidyverse is loaded", {
expect_true( "package:tidyverse" %in% search(),
info = "The tidyverse package should be loaded using library()."
})
})

Using .last_plot() to inspect the most recent plot and compare it to a correct one.

p <- last_plot()

correct_p <- ggplot(prop_usa_winners, aes(decade, proportion)) +
geom_line() + geom_point() +
scale_y_continuous(labels = scales::percent, limits = 0:1, expand = c(0,0))

run_tests({
test_that("The plot uses the correct dataset", {
expect_equivalent(p$data, correct_p$data,
info = "The plot should show the data in prop_usa_winners.")
})
})

There's a nice overview of how to use it under the "testthat package" heading in this "Writing tests" blog post. Here's a key extract from the post, documenting some (but not all) functions:

The testthat package includes a number of helper functions for checking whether results match expectation.

  • expect_error() if the code should throw an error.
  • expect_warning() if the code should throw a warning
  • expect_equal(a, b) if a and b should be the same (up to numeric tolerance)
  • expect_equivalent(a, b) if a and b should contain the same values but may have different attributes (e.g., names and dimnames)

For further information, the testing chapter in the R packages book gets into the details, along with workflow advice and concrete examples: http://r-pkgs.had.co.nz/tests.html

Writing Tests in Python

Overview of testing in Python

Testing in Python is done using the nose framework, which is a Python unit test framework. After the ipython_nose extension is loaded via the first cell using %load_ext ipython_nose, you simply need to begin each @tests cell with %%nose and then include your tests.

There are instructions in each notebook for installing nose and the ipython_nose extension so that you can run your tests locally.

Examples of testing in Python

Below are some examples of nose tests from our A Visual History of Nobel Prize Winners (Python) Guided Project.

Test a package has been loaded and aliased correctly.

def test_pandas_loaded():
assert pd.__name__ == 'pandas', \
"pandas should be imported as pd"

Test the output of a student to the correct output by using an underscore to refer to the last expression value.

last_value = _

correct_value = nobel['birth_country'].value_counts().head(10)

def test_last_value_correct():
assert last_value.equals(correct_value), \
"The number of prizes won by the top 10 nationalities doesn't seem correct... Maybe check the hint?"

Test that the y-axis data from a plot is the correct data.

def test_y_axis():
assert all(ax.get_lines()[0].get_ydata() == prop_usa_winners.usa_born_winner), \
'The plot should be assigned to ax and have usa_born_winner on the y-axis'

Testing for Guided Projects

For DataCamp projects as a whole, we don't need to and can't possibly test everything. We just need to give students some help along the way. It is difficult/impossible to test things not saved to objects and cells with multiple plots. And that's okay!

However, testing should be rigorous enough to ensure that interdependent cells still function correctly. If a student needs to use a DataFrame throughout a project, there should be sufficient testing to ensure that they don’t arrive at a later task with incorrect data.

Testing for Unguided Projects

Testing for Unguided Projects is relatively the same as it is for Guided Projects (i.e. using nose and testthat), but as the focus on these projects is the outcome, rather than the solution, they should be considerably shorter and less focused on the methodology.

However, it is also generally best practice to design Unguided Project tests with the following considerations:

  • Are there alternate solutions that are still correct, but will produce different answers? If so, the test should be written to accept additional viable answers.
  • Can you diagnose common errors/problems? For example, if missing a cleaning step will produce an incorrect answer, you can first compare the learner solution with that result (e.g. learner_solution != missed_cleaning), and include the relevant feedback message.
  • Are there additional checks you can perform on the answer to help steer students in the correct direction and prevent simple mistakes (e.g. data type checks, DataFrame size, etc).
  • Can you write the tests to be lenient enough to accept answers that are close enough? For example, when testing for a string answer, converting the student solution to lower-case prior to testing.

Testing your tests

You can run your tests in your Jupyter Notebook by running the solution code then the tests back to back. Learn more in this article on checking your tests.

If you are stuck, don’t hesitate to reach out via email ([email protected]), Asana, or here on GitHub (@datacamp-contentquality)!

Did this answer your question?