Quickstart

Contents

!pip install aiohttp matplotlib pandas pyarrow requests "xarray[io]"

Show code cell output

Hide code cell output

Requirement already satisfied: aiohttp in /home/deploy/.local/lib/python3.8/site-packages (3.10.11)

Requirement already satisfied: matplotlib in /usr/lib/python3/dist-packages (3.1.2)
Requirement already satisfied: pandas in /home/deploy/.local/lib/python3.8/site-packages (2.0.3)

Requirement already satisfied: pyarrow in /home/deploy/.local/lib/python3.8/site-packages (17.0.0)
Requirement already satisfied: requests in /usr/lib/python3/dist-packages (2.22.0)
Requirement already satisfied: xarray[io] in /home/deploy/.local/lib/python3.8/site-packages (2023.1.0)

Requirement already satisfied: yarl<2.0,>=1.12.0 in /home/deploy/.local/lib/python3.8/site-packages (from aiohttp) (1.15.2)
Requirement already satisfied: attrs>=17.3.0 in /usr/lib/python3/dist-packages (from aiohttp) (19.3.0)
Requirement already satisfied: multidict<7.0,>=4.5 in /home/deploy/.local/lib/python3.8/site-packages (from aiohttp) (6.1.0)
Requirement already satisfied: frozenlist>=1.1.1 in /home/deploy/.local/lib/python3.8/site-packages (from aiohttp) (1.5.0)
Requirement already satisfied: aiosignal>=1.1.2 in /home/deploy/.local/lib/python3.8/site-packages (from aiohttp) (1.3.1)
Requirement already satisfied: async-timeout<6.0,>=4.0; python_version < "3.11" in /home/deploy/.local/lib/python3.8/site-packages (from aiohttp) (5.0.1)
Requirement already satisfied: aiohappyeyeballs>=2.3.0 in /home/deploy/.local/lib/python3.8/site-packages (from aiohttp) (2.4.4)
Requirement already satisfied: tzdata>=2022.1 in /home/deploy/.local/lib/python3.8/site-packages (from pandas) (2025.2)
Requirement already satisfied: pytz>=2020.1 in /home/deploy/.local/lib/python3.8/site-packages (from pandas) (2025.2)
Requirement already satisfied: python-dateutil>=2.8.2 in /home/deploy/.local/lib/python3.8/site-packages (from pandas) (2.9.0.post0)
Requirement already satisfied: numpy>=1.20.3; python_version < "3.10" in /home/deploy/.local/lib/python3.8/site-packages (from pandas) (1.24.4)
Requirement already satisfied: packaging>=21.3 in /usr/local/lib/python3.8/dist-packages (from xarray[io]) (24.1)
Requirement already satisfied: netCDF4; extra == "io" in /home/deploy/.local/lib/python3.8/site-packages (from xarray[io]) (1.7.2)
Requirement already satisfied: zarr; extra == "io" in /home/deploy/.local/lib/python3.8/site-packages (from xarray[io]) (2.16.1)

Requirement already satisfied: rasterio; extra == "io" in /home/deploy/.local/lib/python3.8/site-packages (from xarray[io]) (1.3.11)
Requirement already satisfied: pydap; python_version < "3.10" and extra == "io" in /home/deploy/.local/lib/python3.8/site-packages (from xarray[io]) (3.4.1)

Requirement already satisfied: cftime; extra == "io" in /home/deploy/.local/lib/python3.8/site-packages (from xarray[io]) (1.6.4.post1)
Requirement already satisfied: fsspec; extra == "io" in /home/deploy/.local/lib/python3.8/site-packages (from xarray[io]) (2024.10.0)

Requirement already satisfied: h5netcdf; extra == "io" in /home/deploy/.local/lib/python3.8/site-packages (from xarray[io]) (1.1.0)
Requirement already satisfied: pooch; extra == "io" in /home/deploy/.local/lib/python3.8/site-packages (from xarray[io]) (1.8.2)

Requirement already satisfied: scipy; extra == "io" in /home/deploy/.local/lib/python3.8/site-packages (from xarray[io]) (1.10.1)
Requirement already satisfied: cfgrib; extra == "io" in /home/deploy/.local/lib/python3.8/site-packages (from xarray[io]) (0.9.15.0)
Requirement already satisfied: propcache>=0.2.0 in /home/deploy/.local/lib/python3.8/site-packages (from yarl<2.0,>=1.12.0->aiohttp) (0.2.0)
Requirement already satisfied: idna>=2.0 in /usr/lib/python3/dist-packages (from yarl<2.0,>=1.12.0->aiohttp) (2.8)
Requirement already satisfied: typing-extensions>=4.1.0; python_version < "3.11" in /home/deploy/.local/lib/python3.8/site-packages (from multidict<7.0,>=4.5->aiohttp) (4.13.2)
Requirement already satisfied: six>=1.5 in /usr/lib/python3/dist-packages (from python-dateutil>=2.8.2->pandas) (1.14.0)
Requirement already satisfied: certifi in /usr/lib/python3/dist-packages (from netCDF4; extra == "io"->xarray[io]) (2019.11.28)
Requirement already satisfied: fasteners in /home/deploy/.local/lib/python3.8/site-packages (from zarr; extra == "io"->xarray[io]) (0.19)
Requirement already satisfied: numcodecs>=0.10.0 in /home/deploy/.local/lib/python3.8/site-packages (from zarr; extra == "io"->xarray[io]) (0.12.1)
Requirement already satisfied: asciitree in /home/deploy/.local/lib/python3.8/site-packages (from zarr; extra == "io"->xarray[io]) (0.3.3)
Requirement already satisfied: importlib-metadata; python_version < "3.10" in /usr/lib/python3/dist-packages (from rasterio; extra == "io"->xarray[io]) (1.5.0)
Requirement already satisfied: affine in /home/deploy/.local/lib/python3.8/site-packages (from rasterio; extra == "io"->xarray[io]) (2.4.0)

Requirement already satisfied: cligj>=0.5 in /home/deploy/.local/lib/python3.8/site-packages (from rasterio; extra == "io"->xarray[io]) (0.7.2)
Requirement already satisfied: setuptools in /usr/lib/python3/dist-packages (from rasterio; extra == "io"->xarray[io]) (45.2.0)
Requirement already satisfied: click-plugins in /home/deploy/.local/lib/python3.8/site-packages (from rasterio; extra == "io"->xarray[io]) (1.1.1.2)
Requirement already satisfied: click>=4.0 in /usr/lib/python3/dist-packages (from rasterio; extra == "io"->xarray[io]) (7.0)
Requirement already satisfied: snuggs>=1.4.1 in /home/deploy/.local/lib/python3.8/site-packages (from rasterio; extra == "io"->xarray[io]) (1.4.7)
Requirement already satisfied: docopt in /usr/lib/python3/dist-packages (from pydap; python_version < "3.10" and extra == "io"->xarray[io]) (0.6.2)
Requirement already satisfied: Webob in /home/deploy/.local/lib/python3.8/site-packages (from pydap; python_version < "3.10" and extra == "io"->xarray[io]) (1.8.9)
Requirement already satisfied: Jinja2 in /usr/local/lib/python3.8/dist-packages (from pydap; python_version < "3.10" and extra == "io"->xarray[io]) (3.1.4)
Requirement already satisfied: beautifulsoup4 in /home/deploy/.local/lib/python3.8/site-packages (from pydap; python_version < "3.10" and extra == "io"->xarray[io]) (4.13.4)
Requirement already satisfied: h5py in /home/deploy/.local/lib/python3.8/site-packages (from h5netcdf; extra == "io"->xarray[io]) (3.11.0)
Requirement already satisfied: platformdirs>=2.5.0 in /home/deploy/.local/lib/python3.8/site-packages (from pooch; extra == "io"->xarray[io]) (4.3.6)
Requirement already satisfied: eccodes>=0.9.8 in /home/deploy/.local/lib/python3.8/site-packages (from cfgrib; extra == "io"->xarray[io]) (2.42.0)
Requirement already satisfied: pyparsing>=2.1.6 in /usr/lib/python3/dist-packages (from snuggs>=1.4.1->rasterio; extra == "io"->xarray[io]) (2.4.6)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.8/dist-packages (from Jinja2->pydap; python_version < "3.10" and extra == "io"->xarray[io]) (2.1.5)
Requirement already satisfied: soupsieve>1.2 in /home/deploy/.local/lib/python3.8/site-packages (from beautifulsoup4->pydap; python_version < "3.10" and extra == "io"->xarray[io]) (2.7)
Requirement already satisfied: cffi in /home/deploy/.local/lib/python3.8/site-packages (from eccodes>=0.9.8->cfgrib; extra == "io"->xarray[io]) (1.17.1)
Requirement already satisfied: findlibs in /home/deploy/.local/lib/python3.8/site-packages (from eccodes>=0.9.8->cfgrib; extra == "io"->xarray[io]) (0.1.1)
Requirement already satisfied: pycparser in /home/deploy/.local/lib/python3.8/site-packages (from cffi->eccodes>=0.9.8->cfgrib; extra == "io"->xarray[io]) (2.22)

import pandas as pd
URL = 'https://mastapp.site'

Quickstart#

This notebook shows some very simple example of getting started.

List All Shots#

shots_df = pd.read_parquet(f'{URL}/parquet/level2/shots')
shots_df

	uuid	equi_max_li3	ohmnic_max_heating	generic_max_energy_time	generic_min_q95_time	endpoint_url	generic_max_geo_major_radius_time	timestamp	shot_postshot_comment	generic_max_beta_poloidal_mhd	...	nbi_power_max_ss	nbi_energy_ss_max_power	nbi_max_ss_power_time	nbi_power_ss_max_current	nbi_power_truby_ss	scenario	rad_o2ratio	radii_c2ratio	shot_scenario	shot_flat_top_duration
0	7da2590b-6b7c-5a4e-8125-8b0c38a7c8bf	1.520526	1.038769	0.037496	0.270	https://s3.echo.stfc.ac.uk	0.080	2005-01-13 12:02:00	GOOD PLASMA, RAN FINE. SL JOINT ALARMS RATHER ...	0.075	...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	None	NaN
1	594cabe7-8b0a-54a2-9e1e-8b250a858acd	1.217211	1.105280	0.019210	0.245	https://s3.echo.stfc.ac.uk	0.080	2005-01-13 12:17:00	OK BUT LOST VERTICAL CONTROL - FA2 JUST DIED A...	0.245	...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	None	NaN
2	23a8941d-e073-5073-8f5e-e53230f55b43	1.262946	0.849237	0.036312	0.150	https://s3.echo.stfc.ac.uk	0.080	2005-01-13 13:30:00	OK. GOT FA4 BUT NOT FA3	0.075	...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	None	NaN
3	e12564d4-f0af-5c8d-b15c-ab59f11144b3	1.519628	2.149353	0.057099	0.290	https://s3.echo.stfc.ac.uk	0.290	2005-01-13 13:44:00	SLIDING JOINT ALARMS A BIT LOWER. PLASMA OK.	0.295	...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	None	NaN
4	1562346b-2097-5d27-8649-07eeb48dc5f7	1.788729	0.824824	0.052359	0.285	https://s3.echo.stfc.ac.uk	0.260	2005-01-13 14:33:00	GOOD PLASMA F/B CONTROL. SLIDING JOINT ALARMS ...	0.260	...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	None	NaN
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
11568	db7bc878-869c-59c6-bcc6-cb28cba69cf9	6.694992	NaN	-1.000000	0.030	https://s3.echo.stfc.ac.uk	0.165	2013-09-27 14:03:00	'Two times lower DD neutron rate than referenc...	0.180	...	2.080261	70.840835	0.12160	2.180491	NaN	3.0	NaN	NaN	S6	NaN
11569	696b1a86-1986-59a4-8cd8-f968dba4ece4	8.221705	NaN	-1.000000	0.030	https://s3.echo.stfc.ac.uk	0.155	2013-09-27 14:21:00	'Good beam.Good repeat.'	0.130	...	2.083544	70.921546	0.13065	1.985447	NaN	2.0	NaN	NaN	S8	NaN
11570	dcc5091c-ba43-599e-9c63-f8cb41494a60	2.542882	NaN	-1.000000	0.195	https://s3.echo.stfc.ac.uk	0.170	2013-09-27 14:39:00	'Good shot. Modes present.'	0.175	...	2.163367	75.050461	0.19560	2.288206	NaN	3.0	NaN	NaN	S6	NaN
11571	63ccd361-c339-5403-abf3-dec0ad9d4811	11.771064	10.391692	0.008542	0.035	https://s3.echo.stfc.ac.uk	0.170	2013-09-27 15:03:00	'No HF gas.'	0.060	...	2.204641	75.045603	0.18445	2.107413	NaN	2.0	NaN	NaN	S8	NaN
11572	0916ffb1-fece-5d3d-81a9-b0504f49929f	59.918600	73.787904	0.021805	0.035	https://s3.echo.stfc.ac.uk	0.125	2013-09-27 15:20:00	'Good shot.'	0.060	...	2.062899	71.177179	0.12170	NaN	NaN	2.0	NaN	NaN	S8	NaN

11573 rows × 189 columns

List All Sources#

sources_df = pd.read_parquet(f"{URL}/parquet/level2/sources")
sources_df

	quality	context_	title	name	endpoint_url	imas	description	type_	uuid	url	shot_id
0	Not Checked	{'dcat': 'http://www.w3.org/ns/dcat#', 'dct': ...	Source Dataset	equilibrium	https://s3.echo.stfc.ac.uk	equilibrium		dcat:Dataset	0f3fbb3a-8915-593b-8a5e-bb2801bd5b6d	s3://mast/level2/shots/11766.zarr	11766
1	Not Checked	{'dcat': 'http://www.w3.org/ns/dcat#', 'dct': ...	Source Dataset	gas_injection	https://s3.echo.stfc.ac.uk	gas_injection		dcat:Dataset	21df071f-d08f-540c-8f39-a799dc34535c	s3://mast/level2/shots/11766.zarr	11766
2	Not Checked	{'dcat': 'http://www.w3.org/ns/dcat#', 'dct': ...	Source Dataset	magnetics	https://s3.echo.stfc.ac.uk	magnetics		dcat:Dataset	c97c0e5c-c26c-52af-9b37-f6660d08726d	s3://mast/level2/shots/11766.zarr	11766
3	Not Checked	{'dcat': 'http://www.w3.org/ns/dcat#', 'dct': ...	Source Dataset	pf_active	https://s3.echo.stfc.ac.uk	pf_active		dcat:Dataset	ba47b2a2-a6da-5d8e-b6c2-e093fbfc3553	s3://mast/level2/shots/11766.zarr	11766
4	Not Checked	{'dcat': 'http://www.w3.org/ns/dcat#', 'dct': ...	Source Dataset	pulse_schedule	https://s3.echo.stfc.ac.uk	pulse_schedule		dcat:Dataset	46662499-0e12-5e7a-a8b8-ec44cdfca24d	s3://mast/level2/shots/11766.zarr	11766
...	...	...	...	...	...	...	...	...	...	...	...
105993	Not Checked	{'dcat': 'http://www.w3.org/ns/dcat#', 'dct': ...	Source Dataset	gas_injection	https://s3.echo.stfc.ac.uk	gas_injection		dcat:Dataset	9d2c7bbd-8592-5e3b-a358-921879f34bd4	s3://mast/level2/shots/30471.zarr	30471
105994	Not Checked	{'dcat': 'http://www.w3.org/ns/dcat#', 'dct': ...	Source Dataset	magnetics	https://s3.echo.stfc.ac.uk	magnetics		dcat:Dataset	7118a52b-91bc-5575-ae9d-388552642b72	s3://mast/level2/shots/30471.zarr	30471
105995	Not Checked	{'dcat': 'http://www.w3.org/ns/dcat#', 'dct': ...	Source Dataset	pf_active	https://s3.echo.stfc.ac.uk	pf_active		dcat:Dataset	8a94267d-3c38-516b-83ee-27ba71348716	s3://mast/level2/shots/30471.zarr	30471
105996	Not Checked	{'dcat': 'http://www.w3.org/ns/dcat#', 'dct': ...	Source Dataset	spectrometer_visible	https://s3.echo.stfc.ac.uk	None		dcat:Dataset	7d455235-58fb-5918-9d04-0653d4972b89	s3://mast/level2/shots/30471.zarr	30471
105997	Not Checked	{'dcat': 'http://www.w3.org/ns/dcat#', 'dct': ...	Source Dataset	summary	https://s3.echo.stfc.ac.uk	summary		dcat:Dataset	5da1f1a6-c70f-5ad6-97c4-8e3b1f91da04	s3://mast/level2/shots/30471.zarr	30471

105998 rows × 11 columns

List all signals Given a Shot Number#

signals_df = pd.read_parquet(f"{URL}/parquet/level2/signals?shot_id=30421")
signals_df

	shape	uuid	imas	context_	name	shot_id	version	description	rank	dimensions	url	quality	endpoint_url	type_	source	title	units
0	[73, 160]	75756c34-c5b1-5e75-a53e-82bd90635d75	charge_exchange	{'dcat': 'http://www.w3.org/ns/dcat#', 'dct': ...	t_i	30421	0		2	[time, major_radius]	s3://mast/level2/shots/30421.zarr	Not Checked	https://s3.echo.stfc.ac.uk	dcat:Dataset	charge_exchange	Signal Dataset	eV
1	[146]	883d84bf-823b-54d2-9036-9715ed6e8878	equilibrium	{'dcat': 'http://www.w3.org/ns/dcat#', 'dct': ...	q95	30421	0		1	[time]	s3://mast/level2/shots/30421.zarr	Not Checked	https://s3.echo.stfc.ac.uk	dcat:Dataset	equilibrium	Signal Dataset
2	[146]	bad7b1a8-df4c-5304-99bf-7a4ff92c0cf1	equilibrium	{'dcat': 'http://www.w3.org/ns/dcat#', 'dct': ...	q_axis	30421	0		1	[time]	s3://mast/level2/shots/30421.zarr	Not Checked	https://s3.echo.stfc.ac.uk	dcat:Dataset	equilibrium	Signal Dataset
3	[146]	cceb099c-a98c-5448-bff8-f63b3e5fe6db	equilibrium	{'dcat': 'http://www.w3.org/ns/dcat#', 'dct': ...	rpsi100_in	30421	0		1	[time]	s3://mast/level2/shots/30421.zarr	Not Checked	https://s3.echo.stfc.ac.uk	dcat:Dataset	equilibrium	Signal Dataset
4	[146]	2c4828a9-5b27-54ea-b58a-51bca2a68d59	equilibrium	{'dcat': 'http://www.w3.org/ns/dcat#', 'dct': ...	rpsi100_out	30421	0		1	[time]	s3://mast/level2/shots/30421.zarr	Not Checked	https://s3.echo.stfc.ac.uk	dcat:Dataset	equilibrium	Signal Dataset
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
68	[36321]	d4c4ea86-d28f-50fd-a137-091652b5afba	None	{'dcat': 'http://www.w3.org/ns/dcat#', 'dct': ...	density_gradient	30421	0		1	[time]	s3://mast/level2/shots/30421.zarr	Not Checked	https://s3.echo.stfc.ac.uk	dcat:Dataset	spectrometer_visible	Signal Dataset	V
69	[32, 1452801]	1dccaa5c-4778-56d5-af6a-1a09820f8a67	None	{'dcat': 'http://www.w3.org/ns/dcat#', 'dct': ...	filter_spectrometer_bes_voltage	30421	0		2	[bes_channel, time_bes]	s3://mast/level2/shots/30421.zarr	Not Checked	https://s3.echo.stfc.ac.uk	dcat:Dataset	spectrometer_visible	Signal Dataset	V
70	[3, 36321]	9f3ea8be-e2b1-5806-b9b5-d9606f169e0d	None	{'dcat': 'http://www.w3.org/ns/dcat#', 'dct': ...	filter_spectrometer_dalpha_voltage	30421	0		2	[dalpha_channel, time]	s3://mast/level2/shots/30421.zarr	Not Checked	https://s3.echo.stfc.ac.uk	dcat:Dataset	spectrometer_visible	Signal Dataset	V
71	[2906]	b9b4510f-fec2-5eab-963d-b8086aaf8683	summary	{'dcat': 'http://www.w3.org/ns/dcat#', 'dct': ...	ip	30421	0		1	[time]	s3://mast/level2/shots/30421.zarr	Not Checked	https://s3.echo.stfc.ac.uk	dcat:Dataset	summary	Signal Dataset	A
72	[2906]	f1098e0f-003f-5bc9-a1c5-5e3058d5ddea	summary	{'dcat': 'http://www.w3.org/ns/dcat#', 'dct': ...	line_average_n_e	30421	0		1	[time]	s3://mast/level2/shots/30421.zarr	Not Checked	https://s3.echo.stfc.ac.uk	dcat:Dataset	summary	Signal Dataset	A

73 rows × 17 columns

Open and plot shot data.#

Here we are visualising some plasma current data.

import xarray as xr
import matplotlib.pyplot as plt


dataset = xr.open_zarr(
    "https://s3.echo.stfc.ac.uk/mast/level2/shots/30421.zarr",
    group='summary'
)

dataset['ip'].plot()
plt.show()

_images/ac8260d392885693b3cc46ec6563cfb639e07d167644a44616bc163e592fe79d.png