Best Practices

Paul Schrimpf

2024-09-23

\[ \def\Er{{\mathrm{E}}} \def\En{{\mathbb{E}_n}} \def\cov{{\mathrm{Cov}}} \def\var{{\mathrm{Var}}} \def\R{{\mathbb{R}}} \def\arg{{\mathrm{arg}}} \newcommand\norm[1]{\left\lVert#1\right\rVert} \def\rank{{\mathrm{rank}}} \newcommand{\inpr}{ \overset{p^*_{\scriptscriptstyle n}}{\longrightarrow}} \def\inprob{{\,{\buildrel p \over \rightarrow}\,}} \def\indist{\,{\buildrel d \over \rightarrow}\,} \DeclareMathOperator*{\plim}{plim} \DeclareMathOperator*{\argmax}{argmax} \DeclareMathOperator*{\argmin}{argmin} \]

Overview

Goals

• Correct
• Maintainable & Extensible
  • Clear
  • Contained
  • Consistent
• Efficient

Tools

Goal	Tool(s)
Correct	tests, static analysis
Maintainable	version control, CI, documentation, dependency management
Efficient	benchmarks, profiler

Version Control

Version Control

• essential
• git is by far the most popular and what I recommend
• git hosting services
  • github.com
  • gitlab.com
  • bitbucket.org
• Benefits:
  • backup
  • edit history
  • coordination

git

• Use it
• git intro and tutorial and webinar

Julia Tools

Tool Setup for Julia

• Organize code into a package
• Good advice aimed at scientific computing projects: Julia: Project Workflow
• Setup your package with a project skeleton
  • PkgTemplates.jl
  • BestieTemplate.jl
  • DoctorWatson.jl
  • Haider, Riesch, and Jirauschek (2021) for similar idea for other languages

PkgTemplates.jl

using PkgTemplates
Template(interactive=true)("SomeNewPackage")

or

using PkgTemplates
tpl = Template(; dir=pwd(),
               user="schrimpf", # github username
               authors=["Paul Schrimpf"],
               plugins=[ProjectFile(), SrcDir(),
                        Tests(project=true, aqua=true, jet=true),
                        License(; name="MIT"),
                        Git(),
                        GitHubActions(),
                        Codecov(),
                        Citation(),
                        Documenter{GitHubActions}(),
                        PkgBenchmark(),
                        Formatter()]
               )
tpl("SomeNewPackage")

[ Info: Running prehooks
[ Info: Running hooks
  Activating new project at `~/compecon/ECON622/qmd/SomeNewPackage/test`
   Resolving package versions...
    Updating `~/compecon/ECON622/qmd/SomeNewPackage/test/Project.toml`
  [8dfed614] + Test
    Updating `~/compecon/ECON622/qmd/SomeNewPackage/test/Manifest.toml`
  [2a0f44e3] + Base64
  [b77e0a4c] + InteractiveUtils
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [9a3f8284] + Random
  [ea8e919c] + SHA v0.7.0
  [9e88b42a] + Serialization
  [8dfed614] + Test
  Activating project at `~/compecon/ECON622/qmd`
  Activating project at `~/compecon/ECON622/qmd/SomeNewPackage/test`
   Resolving package versions...
    Updating `~/compecon/ECON622/qmd/SomeNewPackage/test/Project.toml`
  [4c88cf16] + Aqua v0.8.7
    Updating `~/compecon/ECON622/qmd/SomeNewPackage/test/Manifest.toml`
  [4c88cf16] + Aqua v0.8.7
  [34da2185] + Compat v4.16.0
  [0dad84c5] + ArgTools v1.1.1
  [56f22d72] + Artifacts
  [ade2ca70] + Dates
  [f43a241f] + Downloads v1.6.0
  [7b1f6079] + FileWatching
  [b27032c2] + LibCURL v0.6.4
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [ca575930] + NetworkOptions v1.2.0
  [44cfe95a] + Pkg v1.10.0
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [6462fe0b] + Sockets
  [fa267f1f] + TOML v1.0.3
  [a4e569a6] + Tar v1.10.0
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode
  [deac9b47] + LibCURL_jll v8.4.0+0
  [e37daf67] + LibGit2_jll v1.6.4+0
  [29816b5a] + LibSSH2_jll v1.11.0+1
  [c8ffd9c3] + MbedTLS_jll v2.28.2+1
  [14a3606d] + MozillaCACerts_jll v2023.1.10
  [83775a58] + Zlib_jll v1.2.13+1
  [8e850ede] + nghttp2_jll v1.52.0+1
  [3f19e933] + p7zip_jll v17.4.0+2
  Activating project at `~/compecon/ECON622/qmd`
  Activating project at `~/compecon/ECON622/qmd/SomeNewPackage/test`
   Resolving package versions...
    Updating `~/compecon/ECON622/qmd/SomeNewPackage/test/Project.toml`
  [c3a54625] + JET v0.9.9
    Updating `~/compecon/ECON622/qmd/SomeNewPackage/test/Manifest.toml`
  [da1fd8a2] + CodeTracking v1.3.6
  [c3a54625] + JET v0.9.9
  [aa1ae85d] + JuliaInterpreter v0.9.36
  [6f1432cf] + LoweredCodeUtils v3.0.2
  [1914dd2f] + MacroTools v0.5.13
  [aea7be01] + PrecompileTools v1.2.1
  [21216c6a] + Preferences v1.4.3
  Activating project at `~/compecon/ECON622/qmd`
  Activating new project at `~/compecon/ECON622/qmd/SomeNewPackage/docs`
   Resolving package versions...
   Installed OpenSSL_jll ─ v3.0.15+1
    Updating `~/compecon/ECON622/qmd/SomeNewPackage/docs/Project.toml`
  [e30172f5] + Documenter v1.7.0
    Updating `~/compecon/ECON622/qmd/SomeNewPackage/docs/Manifest.toml`
  [a4c015fc] + ANSIColoredPrinters v0.0.1
  [1520ce14] + AbstractTrees v0.4.5
  [944b1d66] + CodecZlib v0.7.6
  [ffbed154] + DocStringExtensions v0.9.3
  [e30172f5] + Documenter v1.7.0
  [d7ba0133] + Git v1.3.1
  [b5f81e59] + IOCapture v0.2.5
  [692b3bcd] + JLLWrappers v1.6.0
  [682c06a0] + JSON v0.21.4
  [0e77f7df] + LazilyInitializedFields v1.2.2
  [d0879d2d] + MarkdownAST v0.1.2
  [69de0a69] + Parsers v2.8.1
  [aea7be01] + PrecompileTools v1.2.1
  [21216c6a] + Preferences v1.4.3
  [2792f1a3] + RegistryInstances v0.1.0
  [3bb67fe8] + TranscodingStreams v0.11.2
  [2e619515] + Expat_jll v2.6.2+0
  [f8c6e375] + Git_jll v2.44.0+2
  [94ce4f54] + Libiconv_jll v1.17.0+0
  [458c3c95] + OpenSSL_jll v3.0.15+1
  [0dad84c5] + ArgTools v1.1.1
  [56f22d72] + Artifacts
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [f43a241f] + Downloads v1.6.0
  [7b1f6079] + FileWatching
  [b77e0a4c] + InteractiveUtils
  [b27032c2] + LibCURL v0.6.4
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [a63ad114] + Mmap
  [ca575930] + NetworkOptions v1.2.0
  [44cfe95a] + Pkg v1.10.0
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA v0.7.0
  [9e88b42a] + Serialization
  [6462fe0b] + Sockets
  [fa267f1f] + TOML v1.0.3
  [a4e569a6] + Tar v1.10.0
  [8dfed614] + Test
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode
  [deac9b47] + LibCURL_jll v8.4.0+0
  [e37daf67] + LibGit2_jll v1.6.4+0
  [29816b5a] + LibSSH2_jll v1.11.0+1
  [c8ffd9c3] + MbedTLS_jll v2.28.2+1
  [14a3606d] + MozillaCACerts_jll v2023.1.10
  [efcefdf7] + PCRE2_jll v10.42.0+1
  [83775a58] + Zlib_jll v1.2.13+1
  [8e850ede] + nghttp2_jll v1.52.0+1
  [3f19e933] + p7zip_jll v17.4.0+2
Precompiling project...
  ✓ OpenSSL_jll
  ✓ Git_jll
  ✓ Git
  ✓ Documenter
  4 dependencies successfully precompiled in 14 seconds. 19 already precompiled.
   Resolving package versions...
    Updating `~/compecon/ECON622/qmd/SomeNewPackage/docs/Project.toml`
  [eade842b] + SomeNewPackage v1.0.0-DEV `..`
    Updating `~/compecon/ECON622/qmd/SomeNewPackage/docs/Manifest.toml`
  [eade842b] + SomeNewPackage v1.0.0-DEV `..`
  Activating project at `~/compecon/ECON622/qmd`
[ Info: Running posthooks
[ Info: New package is at /home/paul/compecon/ECON622/qmd/SomeNewPackage

"/home/paul/compecon/ECON622/qmd/SomeNewPackage"

BestieTemplate.jl

using BestieTemplate
BestieTemplate.generate("TestPackage")

Tests

• essential
• organize code into small functions, test them all
• test both your code and code from others that you rely on

Documentation

• create it for others and your future self

Continuous Integration

• Automatically execute some actions in the cloud after git commits or pull requests or merges
  • Run tests
  • Build documentation
  • Run static code analysis / Linter
  • Check test coverage
  • etc
• Many providers
  • GitHub Actions
  • TravisCI
  • etc

Test Coverage

• Automatically try to determine which lines of code were executed during testing and produce a summary and report
• app.codecov.io
• coveralls.io

Static Code Analysis

• “linters” analyze code to detect errors and possible bugs
  • built into VSCode and other editors
  • JET.jl for detecting type stability problems (advanced)
• formatters check for following text formatting standards around indentation and such

Further Reading

• Gentzkow and Shapiro (2014)
• Pruim, Gîrjău, and Horton (2023)
• Wilson (2017)
• Haider, Riesch, and Jirauschek (2021)
• Orozco et al. (2020)

References

Gentzkow, Matthew, and Jesse M. Shapiro. 2014. “Code and Data for the Social Sciences: A Practitioner’s Guide.” In. https://api.semanticscholar.org/CorpusID:62408223.

Haider, Michael, Michael Riesch, and Christian Jirauschek. 2021. “Realization of Best Practices in Software Engineering and Scientific Writing Through Ready-to-Use Project Skeletons.” Optical and Quantum Electronics 53 (10): 568. https://doi.org/10.1007/s11082-021-03192-4.

Orozco, Valérie, Christophe Bontemps, Elise Maigné, Virginie Piguet, Annie Hofstetter, Anne Lacroix, Fabrice Levert, and Jean-Marc Rousselle. 2020. “HOW TO MAKE a PIE: REPRODUCIBLE RESEARCH FOR EMPIRICAL ECONOMICS AND ECONOMETRICS.” Journal of Economic Surveys 34 (5): 1134–69. https://doi.org/https://doi.org/10.1111/joes.12389.

Pruim, Randall J., Maria-Cristiana Gîrjău, and Nicholas Jon Horton. 2023. “Fostering Better Coding Practices for Data Scientists.” https://arxiv.org/abs/2210.03991.

Wilson, Jennifer AND Cranston, Greg AND Bryan. 2017. “Good Enough Practices in Scientific Computing.” PLOS Computational Biology 13 (6): 1–20. https://doi.org/10.1371/journal.pcbi.1005510.