Draft
Conversation
Redo of #42 Original MR on Gitlab: https://gitlab.mpcdf.mpg.de/struphy/struphy/-/merge_requests/734 **Solves the following issue(s):** Closes [#318 ](https://gitlab.mpcdf.mpg.de/struphy/struphy/-/issues/318) **Outline:** **This MR is a big refactoring of the high-level interface of Struphy.** The lower level routines like kernels or Derham stuff are not touched. The goal is to address the disparity between the console interface and the Python API (as used in the notebook tutorials) of Struphy. In order to avoid confusion - as there has been for some users - we will shift towards the API and get rid of some console commands, such as `struphy run`. In this MR, we design a `.py` file as a sort of parameter file for the main point of interaction with the user. This file imports all necessary (mostly new) classes for setting up a model and a simulation. Through an IDE like VScode one can then inspect all possible options of a model by clicking or hovering. This is tied to the second point of this MR - replacing `dicts` with `Classes` wherever possible. This will lead to more Pythonic code and easier documentation and coding. Some of the models I have already ported to the new standard. To get familiar with the new ideas, the easiest way is to generate a default `.py` parameter file via ``` struphy params Vlasov ``` This will generate `params_Vlasov.py` in the current working dir. Try to inspect it in an advanced editor like VCcode in order to grasp the new concept. You can run the model via ``` python params_Vlasov.py ``` Parallel runs are started with ``` mpirun -n 2 python params_Vlasov.py ``` Line profiling can be turned on with ``` LINE_PROFILE=1 mpirun -n 2 python params_Vlasov.py ``` Moreover, three tutorials have been ported and moved from `doc/tutorials` to `tutorials/`. You can check these out to learn more about some of the new classes used in the parameter file. **TODOS:** **This MR is now open for improvements from anybody.** We shall discuss in the next meetings. **I have protected this branch - please checkout from this branch and merge back into it!** Models that have already been ported can be seen below. You can try these out as described above. Additionally, the following model tests can be performed: ``` python src/struphy/models/tests/test_Maxwell.py python src/struphy/models/tests/test_LinearMHD.py ``` **Workflow:** * I suggest that everybody tries to port his/her model to the new framework, see the unresolved threads below. * Reminder: **I have protected this branch - please checkout from this branch and merge back into it!** * Once you have ported a model, please add it to the list of models to be tested, here: https://gitlab.mpcdf.mpg.de/struphy/struphy/-/blob/318-parameter-file-as-py/src/struphy/models/tests/test_models.py?ref_type=heads#L16 * Any help is welcome - thanks! Stefan --------- Co-authored-by: Stefan Possanner <stefan.possanner@ipp.mpg.de> Co-authored-by: Byung Kyu Na <byungkyu.na@ipp.mpg.de>
Redo of #36 Added trailing commas. This can be merged into devel after 318 is done. --------- Co-authored-by: Stefan Possanner <stefan.possanner@ipp.mpg.de> Co-authored-by: Byung Kyu Na <byungkyu.na@ipp.mpg.de> Co-authored-by: Stefan Possanner <86720346+spossann@users.noreply.github.com>
Update the PR templates again
Redo of #47 Added badges to README and refactored the Github actions a bit so that we use a [reusable workflow](https://github.com/orgs/community/discussions/52616) for each OS.
Redo of #49 **Solves the following issue(s):** Closes #30 Updated psydac dependency to https://github.com/struphy-hub/psydac-for-struphy.git --------- Co-authored-by: Stefan Possanner <stefan.possanner@ipp.mpg.de> Co-authored-by: Byung Kyu Na <byungkyu.na@ipp.mpg.de> Co-authored-by: Stefan Possanner <86720346+spossann@users.noreply.github.com>
Redo of #61 **Solves the following issue(s):** Closes #58 I just ran the following command ``` ruff check --select E713 --fix ``` https://docs.astral.sh/ruff/rules/not-in-test/ --------- Co-authored-by: Stefan Possanner <stefan.possanner@ipp.mpg.de> Co-authored-by: Byung Kyu Na <byungkyu.na@ipp.mpg.de> Co-authored-by: Stefan Possanner <86720346+spossann@users.noreply.github.com>
Redo of #62 **Solves the following issue(s):** Closes #57 I ran the following command: ``` ruff check --select E712 --unsafe-fixes --fix ``` Note the `--unsafe-fixes`, so please check the changes. --------- Co-authored-by: Stefan Possanner <stefan.possanner@ipp.mpg.de> Co-authored-by: Byung Kyu Na <byungkyu.na@ipp.mpg.de> Co-authored-by: Stefan Possanner <86720346+spossann@users.noreply.github.com>
Redo of #63 **Solves the following issue(s):** Closes #56 I ran the following command: ``` ruff check --select F541 --fix ``` https://docs.astral.sh/ruff/rules/f-string-missing-placeholders/ --------- Co-authored-by: Stefan Possanner <stefan.possanner@ipp.mpg.de> Co-authored-by: Byung Kyu Na <byungkyu.na@ipp.mpg.de> Co-authored-by: Stefan Possanner <86720346+spossann@users.noreply.github.com>
Redo of #64 (with a few extra edits of the CI) **Solves the following issue(s):** Closes #54 - Formatted all the `.ipynb` files in the repo - Added the notebooks to the formatting checks in the CI --------- Co-authored-by: Stefan Possanner <stefan.possanner@ipp.mpg.de> Co-authored-by: Byung Kyu Na <byungkyu.na@ipp.mpg.de> Co-authored-by: Stefan Possanner <86720346+spossann@users.noreply.github.com>
Redo of #66, and a few lingering changed which were not included previously. --------- Co-authored-by: Stefan Possanner <stefan.possanner@ipp.mpg.de> Co-authored-by: Byung Kyu Na <byungkyu.na@ipp.mpg.de> Co-authored-by: Stefan Possanner <86720346+spossann@users.noreply.github.com>
Redo of #65 **Solves the following issue(s):** Closes #51 I deleted variables that were unused before redefinition according to `ruff check --select F811`. https://docs.astral.sh/ruff/rules/redefined-while-unused/ The biggest change was the class `PushVinViscousPotential `, which seems to be written twice. I removed the first one, but please can someone check that the version that remains is correct? --------- Co-authored-by: Stefan Possanner <stefan.possanner@ipp.mpg.de> Co-authored-by: Byung Kyu Na <byungkyu.na@ipp.mpg.de> Co-authored-by: Stefan Possanner <86720346+spossann@users.noreply.github.com>
**Solves the following issue(s):** Closes #97 Also set `concurrency` as `cancel-in-progress: true` for the relevant workflows.
**Solves the following issue(s):** Closes #101 The new doc can already be inspected: https://struphy-hub.github.io/struphy/
**Solves the following issue(s):** Closes #108
**Solves the following issue(s):** Closes #67 - [x] publish to docker hub - [x] publish to GHCR The Docker hub images are published here: https://hub.docker.com/u/spossann The GHCR images are here: https://github.com/orgs/struphy-hub/packages?repo_name=struphy Both are updated automatically on push to `main.`
**Solves the following issue(s):** Closes #111
**Solves the following issue(s):** Closes #113 I added a trusted publisher on the struphy project on PyPI. I tested the workflow ` pypi-release.yml` on test.pypi: https://test.pypi.org/project/struphy-test-4/2.6.2/#description
This is a test deployment before Struphy 3.0 - to make sure our workflow is correct. I also changed some of the links in the doc from Gitlab to Github.
- [x] Automatically install psydac on first import of struphy by moving all the psydac install logic from `compile.py` to `__init__.py`. - [x] Remove redundant `struphy compile --status` commands Try it out by removing the psydac dependency and then import struphy with ``` pip install . pip uninstall psydac python -c "import struphy" ``` This should install psydac since it isn't installed, and it isn't found in pyproject.toml --------- Co-authored-by: Stefan Possanner <stefan.possanner@ipp.mpg.de>
A new work flow will be triggered for PRs into `main` which checks (in image `ubunut-with-reqs`): * build struphy * install struphy from built `.whl` * compile struphy * test quickrun from README
**Solves the following issue(s):** Closes #120 --------- Co-authored-by: hattom <2407294+hattom@users.noreply.github.com>
Slight format change in CHANGELOG. Added files for readthedocs: * .readthedocs.yml * doc/requirements.txt
**Solves the following issue(s):** Tests Ubuntu and MacOS on push to `devel` (updates badges). At the moment, only a light version of the test is performed. The tests have still to be adapted to be effective.
Redo of #69 **Solves the following issue(s):** Closes #68 According to [PEP 8](https://peps.python.org/pep-0008/#programming-recommendations), "Comparisons to singletons like None should always be done with is or is not, never the equality operators." This PR fixes Ruff errors [E711](https://docs.astral.sh/ruff/rules/none-comparison/), [E714](https://docs.astral.sh/ruff/rules/not-is-test/), [E721](https://docs.astral.sh/ruff/rules/type-comparison/), and [F722](https://docs.astral.sh/ruff/rules/forward-annotation-syntax-error/).
**Core changes:** Add `set_zero_velocity` argument into `LoadingParameters`, enforcing velocities of all particles along specified axis to always be zero. **Documentation changes:** New argument to `LoadingParameters`
Viscosity tensor finished and tested against analytic result. 2d Test for periodic bc --------- Co-authored-by: Stefan Possanner <stefan.possanner@ipp.mpg.de>
**Solves the following issue(s):** Closes #169 **Core changes:** 1. New class `Simulation` inherits the generic `SimulationBase`, both are in the new folder `struphy/simulations/`. The most import methods are: * `Simulation.run()` * `Simulation.pproc()` * `Simulation.load_plotting_data()` * `Simulation.spawn_sister()` (my new favorite!) These are tested in the new tutorials: https://github.com/struphy-hub/struphy-tutorials/tree/use-species-properties The file `main.py` has been deleted. The `Simulation` takes a model as input. Other API classes are passed as well (see tutorials). The model is viewed as everything related to the PDE, i.e. its variables, initial conditions etc. The simulation deals with the rest (geometry, derham, environment etc.) Some important changes to the logic: The model does not have access to `derham`, `mass_ops` etc. anymore, these can be called from `Propagator` when needed. Solves that need to happen before the time stepping (like initial Poisson solves) are moved to `model.allocate_helpers()`. 2. The default launch file has been improved. See Tutorial 2 or test with `struphy params MODEL`. Thus, the available files in the submodule struphy-parameters must be adapted (TODO). 3. Several new classes have been introduced for post processing and plotting data, see `post_processing_tools.py`. The most important ones are `PostProcessor` and `PlottingData`. Dictionaries in the plotting data have been replaced by classes. Many classes now feature the `__repr__` dunder for customized printing. 4. `verbose: bool = False` has been added everywhere in the simulation methods. The handling of verbosity will need some more improvements in the future though. 5. Disable the `ruff format --check` [40c54be](40c54be) 6. Introduce class properties in Particles classes [554caa2](554caa2) **API changes:** New classes exposed: `Simulation`, `PostProcessor` and `PlottingData`. **Model-specific changes:** None **Documentation changes:** The Quickstart has been adapted to the new classes. **Things left TODO:** - [x] Adapt submodule struphy-parameters - [x] Add docstrings
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* New lower bound on pyccel is set to 2.2.0, due to this pyccel bug fix: pyccel/pyccel#2567 * Doc and README: pull docker images from docker hub, which does not require a password.
Update the readme with ``` quarto render README.qmd -M title="" ``` (the `-M title=""` is added to avoid adding a heading at the start of the readme file). This PR also adds a `show_domain` method to the Simulation class using pyvista. Check the new readme [here](https://github.com/struphy-hub/struphy/tree/quarto-readme).
**Solves the following issue(s):** - Add `to_dict` and `from_dict`methods to the Simulation class and other parameter and core classes to enable serialization and deserialization of simulation configurations and objects. This allows saving, loading, and transferring simulation state and parameters in a structured way. - Add `__eq__` method to the Simulation class Example: ```python from struphy import Simulation from struphy.models import Maxwell sim1 = Simulation(model=Maxwell()) sim2 = Simulation.from_dict(sim1.to_dict()) assert sim1 == sim2 ```
* New classmethod `StruphyModel.info()` that shows the html docstring [(see for instance updated tutorials 01 and 05)](https://github.com/struphy-hub/struphy-tutorials/tree/add-model-info) * Helper methods for translating rst to markdown and html * The conventional rst docstring now appears under `__doc_rst__`, which is processed by Sphinx. A custom extension has been written for this. In practice, we only write the `__doc_rst__` string and then auto-translate to html, which becomes the class docstring under the class name (this is seen by Pylance). Models with improved docstrings so far: - [x] Maxwell - [x] VlasovAmpereOneSpecies - [x] LinearMHD --------- Co-authored-by: Max <max.lindqvist@ipp.mpg.de>
This PR is based on the idea that `__repr__` should [return a printable representation of an](https://www.geeksforgeeks.org/python/python-__repr__-magic-method/). The goal is for all objects in the API to have this method, so that a simulation setup can be reproduced using a new `generate_script()` or `save_script(filepath)` method. - Added `__repr__` and `__repr_no_defaults__` methods to most classes in the API. This allows us to create a repr of the instance without cluttering it with default parameters. - Renamed a few old `__repr__` methods that would fit better as `__str__` - Updated tutorials by using `print(x)` instead of `x` - Added `generate_script()` and `save_script(filepath)` methods to the Simulation class - The generated strings are formatted with ruff, which also removes unused imports. Therefore, ruff is moved from a dev dependency to a normal dependency. This is the easiest way to use a consistent formatting for the generated scripts. - Added test to compare the script generation from two simulations.
The weights in models with an initial Poisson solve are set to zero for noise reduction (automatic control variate). In runs without control variate, they must be restored after the initial Poisson solve.
- Added a few new methods for viewing and exporting 3D views of the domains - `create_geometry_mesh`: Generates a pyvista mesh - `show_3d`: Displays the 3D geometry using pyvista (works in console and notebooks) - `export_geometry`: Save the mesh as `vts` or `vtp`. - Added an iterator to the domain class so that we can loop over all domains - Added 3D views for tutorial 6 - Added the neccesary dependencies. You can now view the domain with: ```python from struphy import domains domain = domains.Tokamak() domain.show_3d() ``` <img width="528" height="427" alt="image" src="https://github.com/user-attachments/assets/bb0dc67e-5577-4d27-9486-ba102f470d44" /> To display all domains (except for the Spline ones, which don't work with default params) you can do this: ```python from struphy.geometry.base import Domain for domain_cls in Domain: if "Spline" in domain_cls.__name__: continue print(domain_cls.__name__) domain = domain_cls() domain.show_3d() ```
Added `export` and `from_file` methods to the `SimulationBase` class.
Now all the following examples work:
```python
from struphy import Simulation
from struphy.models import Maxwell
sim1 = Simulation(model=Maxwell())
# Test 1 - to_dict/from_dict
dct = sim1.to_dict()
sim2 = Simulation.from_dict(dct)
assert sim1 == sim2
# Test 2 - export/import YAML
sim1.export("sim1.yaml")
sim3 = Simulation.from_file("sim1.yaml")
assert sim1 == sim3
# Test 3 - export/import JSON
sim1.export("sim1.json")
sim4 = Simulation.from_file("sim1.json")
assert sim1 == sim4
```
Example:
```python
from struphy import Simulation
from struphy.models import Maxwell
sim = Simulation(
model=Maxwell(),
name="Test Simulation",
description="This is standard Maxwell simulation.",
)
```
This makes looping over the models easier, it can now be done with:
```python
from struphy.models.base import StruphyModel
for model in StruphyModel:
print(model.__name__)
```
Output
```
ColdPlasma
ColdPlasmaVlasov
DeterministicParticleDiffusion
DriftKineticElectrostaticAdiabatic
GuidingCenter
HasegawaWakatani
LinearExtendedMHDuniform
LinearMHD
LinearMHDDriftkineticCC
LinearMHDVlasovCC
LinearMHDVlasovPC
LinearVlasovAmpereOneSpecies
Maxwell
Poisson
PressureLessSPH
RandomParticleDiffusion
ShearAlfven
TwoFluidQuasiNeutralToy
VariationalBarotropicFluid
VariationalCompressibleFluid
VariationalPressurelessFluid
ViscoResistiveDeltafMHD
ViscoResistiveDeltafMHD_with_q
ViscoResistiveLinearMHD
ViscoResistiveLinearMHD_with_q
ViscoResistiveMHD
ViscoResistiveMHD_with_p
ViscoResistiveMHD_with_q
ViscousEulerSPH
ViscousFluid
Vlasov
VlasovAmpereOneSpecies
VlasovMaxwellOneSpecies
LinearVlasovMaxwellOneSpecies
```
Instead of just the userguide, we now have two sections: * Userguide * API guide The userguide contains some copy and paste examples, including * Basics * LineaMHD with DESC equil * Kinetic model: two-stream instability The doc files have been cleaned a bit (removed unused, old files). The new doc is here: https://struphy-hub.github.io/struphy/
**Solves the following issue(s):**
Implement new feature to measure violation of Gauss's law
$$\nabla \cdot \mathbf{E} = \frac{\rho}{\epsilon_0} \Rightarrow
\text{error(t)} = \max(|\mathbf{G}^T\mathbf{M}^1\mathbf{e} -
\mathbf{f}^p|)$$
**Core changes:**
Add measurement of gauss error to VlasovMaxwellOneSpecies:
`src/struphy/models/vlasov_maxwell_one_species.py`
**Documentation changes:**
1. New attribute `measure_gauss_error` to VlasovMaxwellOneSpecies model
2. new line in default parameter file of VlasovMaxwellOneSpecies
---------
Co-authored-by: Stefan Possanner <stefan.possanner@ipp.mpg.de>
- This PR cleans up the dict generation for the options classes by introducing an `OptionsBase` class. - Improved the `TensorProductGrid` class by converting lists from the dicts to tuples. This is needed because by default yaml does not support tuples. - Added correct typehints for `TensorProductGrid `
Addresses #203 Auto-generates PRs into `struphy-parameter-files` and `struphy-tutorials` on push to main in case changes exist in the new folders `examples/` or `tutorials/`. Moreover: * New PR-test of examples: test all `*.py` files in new `examples/` folder * Tests use the new option `Simulation.run(one_time_step=True)` for performing just one time step. * New folder `tutorials/`, used in build of the sphinx doc * New PR-test of tutorials in `tutorials/` * use `def main():` in pproc and regression scripts * Auto-generate tag in the two repos when pushing to struphy main * temporarily add the bound jax-finufft<=1.2.0 until desc has a new release Example of PR: struphy-hub/struphy-parameter-files#14 Example of tag: https://github.com/struphy-hub/struphy-parameter-files/tags --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
- [x] Added private runners on TOK, Raven, Viper, and Pitagora. - [x] Add script for loading the recommended module files for each cluster - [x] Add CI job which installs the code on each cluster - [x] Only run the CI job in a weekly scheduled pipeline (?) The `.github/scripts/install_and_test.sh` script is executed only on the private runners, it installs, compiles and tests struphy. It can be executed manually too and should work on Raven/Viper/TOK/Pitagora To simplify the installation on the clusters, I've added the following bash files in `setup/`: - `modules.pitagora.sh` - `modules.raven.sh` - `modules.tok.sh` - `modules.viper.sh` - `modules.sh` The `modules.MACHINE` contain the variables `MODULES_INTEL` and `MODULES_GCC` which lists the modules needed for intel/gcc on the current machine. The `modules.sh` detects which machine we are on, and helps you load or show the modules needed. ``` # Usage: # ./modules.sh [load|display] # load - Loads the required modules for the detected system/compiler # display - Prints the module load command for the detected system/compiler ``` So for example, if you want to load the GCC modules (which is default), you just type: ``` source ./setup/modules.sh load # or source ./setup/modules.sh load gcc ``` To load the intel modules, you type: ``` source ./setup/modules.sh load intel ```
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Some of these domains are definitely experimental, but nice to see that generating new domains works so easily!
I think the biggest annoyance of the domains is the
det_dfwhich in which theargs.kind_mapare impossible to keep track of. There must be a better to do this...Since making new domains is so easy, I think it would make a lot of sense to allow a user to do this themselves, without having to change things deep in struphy.