numpy, pandas and matplotlib in CodeRunner: fixing Jobe's errors

Symptoms and causes

"failed to map segment" and timeouts: the memory limit

Loading numpy's compiled libraries needs more memory than plain Python, and Jobe limits each job's memory:

ImportError: .../numpy/random/_bit_generator.cpython-36m-x86_64-linux-gnu.so: failed to map segment from shared object

In jobeinabox#4 this went away with no memory limit, and Octave's libopenblas.so.0: failed to map segment in jobe#38 has the same cause. Sometimes there is no error at all: in jobe#66, import numpy took 0.17 seconds in a shell and hit the time limit in CodeRunner. Jobe's maintainer traced that to memory too, and pointed at the server's 16 cores, since numpy starts threads per core and each needs memory. That reporter needed 3,000 MB.

To fix it:

1. Open the question's prototype (or the question), choose Customise, then Advanced customisation, and in the Sandbox group set MemLimit (MB). Set it on the prototype rather than on each question, as the maintainer advises.
2. Try 0 (no limit) once to confirm memory is the problem, then settle on a real value. Don't leave 0 in production: one runaway program can make the server thrash (jobe#83).
3. Upgrade Jobe. Its default Python memory limit is 1,500 MB since 2.2.2 (April 2026), and since 2.1.2 it sets OPENBLAS_NUM_THREADS to 4, so numpy no longer starts a thread per core.

The matplotlib MPLCONFIGDIR warning

Importing matplotlib prints this to stderr (jobeinabox#13):

Matplotlib created a temporary config/cache directory at /tmp/matplotlib-b49jl89b because the default path (/home/jobe00/.cache/matplotlib) is not a writable directory; it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory ...

Jobe's users have no home directory, by design, and CodeRunner treats anything on stderr as an error, so a question that otherwise works fails, and a template grader reports "Bad output from grader" (jobe#73). Setting the variable in .bashrc or /etc/profile.d does nothing, because runguard clears the environment before each job. Making /home/jobe00 writable silences it, but the maintainer warns that it lets one student's run leave files for the next.

The fix is a newer Jobe. Since April 2024 runguard gives each run its own MPLCONFIGDIR; 2.1.2 (February 2025) stopped runs sharing that directory, which caused occasional permission errors, and 2.2.1 (September 2025) also sets HOME to it. jobeinabox builds Jobe from its latest source when the image is built, so an old image keeps the old behaviour: pull or rebuild it.

If you can't upgrade yet, set the variable yourself at the top of the question's template:

# Put this at the top of the question's template, before matplotlib is imported
import os, tempfile
if os.environ.get('MPLCONFIGDIR', '/home').startswith('/home'):
    os.environ['MPLCONFIGDIR'] = tempfile.mkdtemp()

"No module named pandas" although it is installed

Jobe runs /usr/bin/python3, or whatever python3_version in app/Config/Jobe.php names. In jobeinabox#8 the packages had gone into a separately built /usr/local/bin/python3. To see which Python your questions use, make a question whose answer prints the interpreter:

import sys
print(sys.executable)

Then install for that interpreter. jobeinabox installs none of numpy, pandas or matplotlib, so a data-science course needs its own image layer. Current Ubuntu marks the system Python as externally managed (PEP 668), so a plain pip install is refused; use the apt packages, or pip with --break-system-packages:

# Install for the interpreter Jobe runs (here /usr/bin/python3)
sudo apt install python3-numpy python3-pandas python3-matplotlib

# or, when you need versions apt doesn't have
sudo /usr/bin/python3 -m pip install --break-system-packages numpy pandas matplotlib

Add an import of each package to the image build, so a broken install fails the build rather than a student's quiz. If you change python3_version, restart the server so Jobe forgets its cached language list.

Or use a server with the stack installed

OnlineCompiler.io's Jobe-compatible server runs CodeRunner's python3 questions on Python 3.14 with numpy 2.5.4, pandas 3.0.6, scipy 1.18.1, scikit-learn 1.9.1, matplotlib 3.11.2, seaborn 0.13.2, statsmodels 0.15.0, sympy 1.14.0, openpyxl 3.1.5, pillow 13.0.0, duckdb 1.5.6. matplotlib needs no setup: import numpy and a script that saves a matplotlib figure both run cleanly through the Jobe API, checked on 2 November 2026. Set CodeRunner's Jobe server to https://api.onlinecompiler.io/api and its Jobe API key to an OnlineCompiler.io API key. The free plan includes 100,000 runs a month.

The limits are ours rather than the question's: 512 MB of memory per run whatever MemLimit says, and up to 30 seconds. That covers teaching-sized numpy and pandas work, but not models that need gigabytes. The Jobe-compatible API page lists every difference from Jobe, and CodeRunner without a Jobe server has the setup.

Frequently asked questions

Why does "import numpy" time out in CodeRunner when it takes a fraction of a second in a shell?

Jobe's per-job memory limit. When numpy cannot get the memory it asks for, the import can fail or spin until the time limit, so CodeRunner reports a timeout. Raise MemLimit in the question's prototype. Jobe's maintainer traced the same symptom in jobe#66 to memory, and numpy starts more threads on machines with many cores.

Can I set MemLimit to 0?

To confirm the limit is the problem, yes. Jobe's maintainer advises against leaving 0 in production, because one program that eats memory can make the whole server thrash. Find the lowest value that works and keep a margin.

Why doesn't setting MPLCONFIGDIR in .bashrc or /etc/profile work?

runguard clears the environment before each job, for security, and keeps only a handful of variables such as PATH and the locale settings. Set it in the question's template, or upgrade Jobe, which now sets it for every run.

Where do I find out which Python Jobe runs?

Run a question whose answer prints sys.executable. Jobe runs /usr/bin/python3 unless python3_version in app/Config/Jobe.php says otherwise, so packages installed into another Python are not found.

Is there a CodeRunner server with numpy and pandas already installed?

Yes. OnlineCompiler.io's Jobe-compatible server runs Python 3.14 with numpy 2.5.4, pandas 3.0.6, scipy 1.18.1, scikit-learn 1.9.1, matplotlib 3.11.2, seaborn 0.13.2, statsmodels 0.15.0, sympy 1.14.0, openpyxl 3.1.5, pillow 13.0.0, duckdb 1.5.6. Set CodeRunner's Jobe server to https://api.onlinecompiler.io/api and its Jobe API key to an OnlineCompiler.io API key; the free plan includes 100,000 runs a month.