Skip to content

Comments

[Experimental] Use jupyter-book 2.0 for building the lectures#363

Open
kp992 wants to merge 29 commits intomainfrom
jb2
Open

[Experimental] Use jupyter-book 2.0 for building the lectures#363
kp992 wants to merge 29 commits intomainfrom
jb2

Conversation

@kp992
Copy link

@kp992 kp992 commented Jul 23, 2025

Experimental setup for building lectures with jupyter-book 2.0.

This PR is built on top of #345.

@github-actions
Copy link

@github-actions github-actions bot temporarily deployed to pull request July 23, 2025 01:19 Inactive
@kp992 kp992 requested a review from mmcky July 23, 2025 01:20
@mmcky
Copy link
Contributor

mmcky commented Jul 23, 2025

@kp992 thanks for opening this. Excellent work! It is exciting to see how close we are now.

From my review it looks like there a just a few issues and features we need before launch.

Bugs:

  1. LHS TOC menu flickers when using the Home page or a top level Part section. It is like it has an auto-stow feature that is triggering when pages are reloaded. It doesn't look great when pages are refreshed.
  2. Code block formatting has too much indentation for Solution blocks

Features:

  1. I see a bunch of new export notebooks being added to the yaml headers of each lecture. Do we need to do some development work on supporting the download notebook link in the theme?
  2. Is index support on the jb2 / mystmd roadmap?

- Add `markdown: commonmark` to all 24 lecture export configs so ipynb
  exports produce plain CommonMark markdown cells compatible with
  vanilla Jupyter Notebook, JupyterLab, and Google Colab
- Add build-ipynb.yml workflow that clones QuantEcon/mystmd@myst-to-ipynb,
  builds from source, exports all ipynb files, and audits for MyST leaks
- All 24 notebooks pass audit (0 MyST syntax leaks)

Uses QuantEcon/mystmd myst-to-ipynb branch which adds:
- CommonMark AST pre-transform (admonitions, math, figures, exercises, etc.)
- Identifier/label stripping to prevent (label)= prefixes
- Image directive stripping for plain ![alt](url) output
- Empty cell filtering and block marker removal
@mmcky
Copy link
Contributor

mmcky commented Feb 25, 2026

CommonMark ipynb Export Support

This commit adds CommonMark notebook export to the JB2 branch, enabling myst build --ipynb to produce notebooks with plain CommonMark markdown cells — compatible with vanilla Jupyter Notebook, JupyterLab (without jupyterlab-myst), and Google Colab.

Changes

24 lecture files — added markdown: commonmark to each export config:

exports:
  - format: ipynb
    markdown: commonmark      # new
    output: exports/<name>.ipynb

New CI workflow (.github/workflows/build-ipynb.yml) — runs in parallel with the existing JB2 workflow:

  • Clones QuantEcon/mystmd@myst-to-ipynb and builds from source
  • Runs myst build --html and myst build --ipynb using the custom CLI
  • Audits all 24 exported notebooks for MyST syntax leaks
  • Uploads ipynb exports as build artifacts

How it works

The markdown: commonmark option triggers an AST pre-transform in myst-to-ipynb that converts MyST-specific nodes to their CommonMark equivalents before serialization:

MyST syntax CommonMark output
:::{note} admonitions > **Note** blockquotes
Math roles $E=mc^2$ dollar signs
Math blocks $$...$$ dollar-dollar
:::{figure} / {image} ![alt](url)
{exercise} / {solution} Exercise N / Solution blocks
{proof} / theorem Theorem (Title) blocks
(label)= targets Stripped
{index} directives Stripped
+++ block markers Stripped

Local test results

All 24 notebooks exported successfully with 0 MyST syntax leaks (verified by automated audit script).

Dependencies

This uses the myst-to-ipynb branch on QuantEcon/mystmd which contains 5 commits:

  • 79c7be0b — Bug fixes (kernelspec, language_info, log message, block markers)
  • f67f1188 — CommonMark serialization mode (~475 lines)
  • c1cca05f — Test suite (35 tests across 3 YAML files)
  • 2d70076d — Real-world validation fixes (identifier stripping, empty cell filter)
  • 2c7fa5d7 — Image directive handler (found during full-project testing)

Add 'images: attachment' to all 24 lecture export configs so that
exported notebooks embed images as base64 cell attachments instead
of referencing external files.

Result: 50 images across 12 notebooks embedded as attachments,
0 external image references remaining. All 24 notebooks validated.
@mmcky
Copy link
Contributor

mmcky commented Feb 25, 2026

@kp992 I have been working on the myst-to-ipynb feature.

Here are a collection of test notebooks: https://github.com/QuantEcon/lecture-python-programming.myst/actions/runs/22381335617 (can download the artifact).

This is looking pretty good I think.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants