Conversation
graph_weather/data/__init__.py unconditionally imported AnemoiDataset, SensorDataset, and WeatherStationReader, each of which pulls in pandas, xarray, anemoi.datasets, or nnja-ai at import time. None of those are declared dependencies of graph_weather, so a plain pip install still crashes on bare import graph_weather even after the nnja_ai guard added to the top-level __init__.py for openclimatefix#179. Wrap each optional submodule import in the same try/except pattern already used at the package root, falling back to None like SensorDataset already does. Add a test that blocks the optional deps and checks graph_weather.data still imports. Follow-up to openclimatefix#179.
|
Dug into the failing checks here. pytest (macos + ubuntu): both jobs crash during test collection, not during any actual test run. The traceback shows a native-library abort in pre-commit.ci: the failing hook is One more thing while I was in here: Net: the two failing checks aren't caused by this branch. Happy to rebase once the segfault/lockfile issue on |
|
Filed #232 for the segfault so it's tracked separately from this PR. |
|
Dug into this failure before pushing anything else. Both jobs fail before any test in this diff runs — pytest crashes during collection with a native segfault: on macOS ( Checked main directly to rule out our diff: same segfault, same Same story for pre-commit.ci: running So both checks are red on main already and this diff doesn't add to either failure. Flagging so it doesn't block review — happy to rebase once the pytest/torch_scatter issue gets fixed upstream. |
Follow-up to #179.
The nnjai_wrapp import from the original report is gone, and
graph_weather/__init__.pyalready guardsfrom .data.nnja_ai import SensorDatasetin a try/except. Butgraph_weather/data/__init__.pystill importsAnemoiDataset,SensorDataset, andWeatherStationReaderunconditionally, and those pull in pandas, xarray, anemoi.datasets, and nnja-ai at import time. None of those are declared dependencies of graph_weather, so a plainpip install graph_weatherstill crashes on bareimport graph_weather- just with a different missing-module error depending on what's installed (pandas, then xarray, then an nnja-ai ImportError).This wraps each of the three imports in
data/__init__.pyin the same try/except pattern already used forSensorDatasetat the package root, falling back toNonewhen the optional dep isn't installed. Also applied the same guard toWeatherStationReaderin the root__init__.py, since it had the same unconditional-import problem.Added a test that blocks pandas/xarray/anemoi.datasets/nnja_ai at import time and checks
graph_weather.datastill imports with all three classes falling back toNone.Verified locally in a clean venv with
pip install --no-deps -e .and none of the optional deps installed -import graph_weather(well,graph_weather.dataspecifically, since I couldn't get a working torch build in this environment) crashed onpandasbefore this change, imports clean after. Couldn't run the full test suite here since I hit the Windows long-path issue installing torch, so I'd appreciate someone running CI on this one rather than trusting my local run alone.