Raw FAIR MAST Data

!pip install pandas matplotlib fsspec s3fs "zarr<3" pyarrow xarray
Hide code cell output 
Requirement already satisfied: pandas in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (2.0.3)
Requirement already satisfied: matplotlib in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (3.9.0)
Requirement already satisfied: fsspec in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (2024.5.0)
Requirement already satisfied: s3fs in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (2024.5.0)
Requirement already satisfied: zarr<3 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (2.15.0)
Requirement already satisfied: pyarrow in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (12.0.1)
Requirement already satisfied: xarray in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (2023.7.0)
Requirement already satisfied: python-dateutil>=2.8.2 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from pandas) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from pandas) (2024.1)
Requirement already satisfied: tzdata>=2022.1 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from pandas) (2024.1)
Requirement already satisfied: numpy>=1.21.0 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from pandas) (1.25.2)
Requirement already satisfied: contourpy>=1.0.1 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from matplotlib) (1.2.1)
Requirement already satisfied: cycler>=0.10 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from matplotlib) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from matplotlib) (4.53.0)
Requirement already satisfied: kiwisolver>=1.3.1 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from matplotlib) (1.4.5)
Requirement already satisfied: packaging>=20.0 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from matplotlib) (24.0)
Requirement already satisfied: pillow>=8 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from matplotlib) (10.3.0)
Requirement already satisfied: pyparsing>=2.3.1 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from matplotlib) (3.1.2)

Requirement already satisfied: aiobotocore<3.0.0,>=2.5.4 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from s3fs) (2.13.0)
Requirement already satisfied: aiohttp!=4.0.0a0,!=4.0.0a1 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from s3fs) (3.9.5)
Requirement already satisfied: asciitree in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from zarr<3) (0.3.3)
Requirement already satisfied: fasteners in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from zarr<3) (0.19)
Requirement already satisfied: numcodecs>=0.10.0 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from zarr<3) (0.12.1)
Requirement already satisfied: botocore<1.34.107,>=1.34.70 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from aiobotocore<3.0.0,>=2.5.4->s3fs) (1.34.106)
Requirement already satisfied: wrapt<2.0.0,>=1.10.10 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from aiobotocore<3.0.0,>=2.5.4->s3fs) (1.16.0)
Requirement already satisfied: aioitertools<1.0.0,>=0.5.1 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from aiobotocore<3.0.0,>=2.5.4->s3fs) (0.11.0)
Requirement already satisfied: aiosignal>=1.1.2 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->s3fs) (1.3.1)
Requirement already satisfied: attrs>=17.3.0 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->s3fs) (23.2.0)
Requirement already satisfied: frozenlist>=1.1.1 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->s3fs) (1.4.1)
Requirement already satisfied: multidict<7.0,>=4.5 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->s3fs) (6.0.5)
Requirement already satisfied: yarl<2.0,>=1.0 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->s3fs) (1.9.4)

Requirement already satisfied: six>=1.5 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from python-dateutil>=2.8.2->pandas) (1.16.0)
Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from botocore<1.34.107,>=1.34.70->aiobotocore<3.0.0,>=2.5.4->s3fs) (1.0.1)
Requirement already satisfied: urllib3!=2.2.0,<3,>=1.25.4 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from botocore<1.34.107,>=1.34.70->aiobotocore<3.0.0,>=2.5.4->s3fs) (2.2.1)
Requirement already satisfied: idna>=2.0 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from yarl<2.0,>=1.0->aiohttp!=4.0.0a0,!=4.0.0a1->s3fs) (3.7)

import zarr
import fsspec
import xarray as xr
import numpy as np
import requests
from scipy.signal import stft
import matplotlib.pyplot as plt
from matplotlib.colors import LogNorm

plt.rcParams["font.family"] = "sans"
plt.rcParams["font.size"] = 8

Raw FAIR MAST Data#

This notebook contains example plots of data from different diagnostics across MAST without any preprocessing, interpolation, calibration, cropping etc. applied to the dataset. Data are supplied under their original names which may be harder to understand and work with. This dataset contains all data traces that could be pulled from the MAST experiment, including instrument calibration and testing shots. This product shot be seen as less reliable.

First we need to find the url to a particular shot. Here we are going to use shot 30421 as an example.

shot_data = requests.get("https://mastapp.site/json/shots/30421")
shot_data = shot_data.json()

fs = fsspec.filesystem(
  **dict(
    protocol='simplecache',
    target_protocol="s3",
    target_options=dict(anon=True, endpoint_url=shot_data['endpoint_url'])
  )
)

store = zarr.storage.FSStore(fs=fs, url=shot_data['url'])

Plasma Current Data#

Data from the amc source contains

  • Plasma Current (\(I_p\)): Flows within the plasma, providing initial heating and contributing to the poloidal magnetic field for confinement and stability.

  • PF Coil Currents: Control the poloidal magnetic field, allowing for plasma shaping, vertical stability, and edge magnetic configuration control.

  • TF Coil Currents: Generate the strong toroidal magnetic field necessary for primary plasma confinement.

dataset = xr.open_zarr(store, group='amc')
dataset = dataset.isel(time=(dataset.time > 0) & (dataset.time < .35))
fig, axes = plt.subplots(3, 1, figsize=(10, 6))
ax1, ax2, ax3 = axes.flatten()

ax1.plot(dataset['time'], dataset['plasma_current'])
ax1.set_xlabel('Time (s)')
ax1.set_ylabel('Plasma Current $I_p$ (kA)')

ax2.plot(dataset['time'], dataset['sol_current'])
ax2.set_xlabel('Time (s)')
ax2.set_ylabel('Solenoid Feed Current (kA)')

ax3.plot(dataset['time'], dataset['tf_current'])
ax3.set_xlabel('Time (s)')
ax3.set_ylabel('TF Feed Current (kA)')

for ax in axes:
    ax.grid(alpha=0.3)

plt.tight_layout()
_images/ae73f4820c731c3242d64a09fe0ffa631f734bf749d93906352975dfc080e664.png

Thompson Scattering Data#

ayc source holds the Thomspon Scattering data at the core. Thomson scattering diagnostics provide accurate measurements of electron temperature and density.

dataset = xr.open_zarr(store, group='ayc')
dataset = dataset[['te_core', 'ne_core']].dropna(dim='time')
fig, axes = plt.subplots(2,1)
ax1, ax2 = axes

ax1.plot(dataset['time'], dataset['te_core'])
ax1.set_xlabel('Time (s)')
ax1.set_ylabel('Core Temperature (eV)')

ax2.plot(dataset['time'], dataset['ne_core'])
ax2.set_xlabel('Time (s)')
ax2.set_ylabel('Peak Core Electron Density ($1 / m^3$)')

for ax in axes:
    ax.grid(alpha=0.3)

plt.tight_layout()
_images/0c7217e84fd003445bb6cd86d41bf7b46bca72e5d03ecc97558d9a6c7354e010.png

CO2 Interferometers#

CO2 interferometers (ane) are used to measure the electron density in the plasma. By measuring the phase shift of the laser beam as it passes through the plasma, the electron density can be inferred with high precision.

dataset = catalog.level1.shots(url=url, group='ane')
dataset = dataset.to_dask()

dataset

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Cell In[6], line 1
----> 1 dataset = catalog.level1.shots(url=url, group='ane')
      2 dataset = dataset.to_dask()
      4 dataset

NameError: name 'catalog' is not defined

dataset = xr.open_zarr(store, group='ane')
plt.plot(dataset['time'], dataset['density'])
ax.set_xlabel('Time (s)')
ax.set_ylabel('Integrated Electron Density ($1 / m^2$)')
ax.grid(alpha=0.3)

plt.tight_layout()
_images/98691808f03df17819867b84aeffe5441183a47016edb94685f57e649329150a.png

Equillibrium Reconstruction Data#

The source efm contains data from EFIT. EFIT is a computational tool used to reconstruct the magnetic equilibrium configuration of the plasma in a tokamak. It calculates the shape and position of the plasma, as well as the distribution of the current and pressure within it, based on magnetic measurements.

dataset = xr.open_zarr(store, group='efm')
dataset

<xarray.Dataset> Size: 8MB
Dimensions:            (time: 120, psi_norm: 65, n_iterations: 10,
                        fcoil_seg_n: 938, fcoil_n: 101, ffprime_coefs_n: 2,
                        mag_probe_n: 78, psi_loop_n: 46, r: 65, z: 65,
                        profile_r: 129, lcfs_coords: 139, limiter_n: 37,
                        pprime_coefs_n: 2, profile_z: 65)
Coordinates: (12/13)
  * fcoil_n            (fcoil_n) float32 404B 0.0 1.0 2.0 ... 98.0 99.0 100.0
  * ffprime_coefs_n    (ffprime_coefs_n) float32 8B 0.0 1.0
  * lcfs_coords        (lcfs_coords) float32 556B 0.0 1.0 2.0 ... 137.0 138.0
  * mag_probe_n        (mag_probe_n) float32 312B 0.0 1.0 2.0 ... 75.0 76.0 77.0
  * n_iterations       (n_iterations) float32 40B 0.0 1.0 2.0 ... 7.0 8.0 9.0
  * pprime_coefs_n     (pprime_coefs_n) float32 8B 0.0 1.0
    ...                 ...
  * profile_z          (profile_z) float32 260B -2.0 -1.938 -1.875 ... 1.938 2.0
  * psi_loop_n         (psi_loop_n) float32 184B 0.0 1.0 2.0 ... 43.0 44.0 45.0
  * psi_norm           (psi_norm) float32 260B 0.0 0.01562 ... 0.9844 1.0
  * r                  (r) float32 260B 0.06 0.09031 0.1206 ... 1.939 1.97 2.0
  * time               (time) float32 480B -0.05 -0.045 -0.04 ... 0.56 0.565 0.6
  * z                  (z) float32 260B -2.0 -1.938 -1.875 ... 1.875 1.938 2.0
Dimensions without coordinates: fcoil_seg_n, limiter_n
Data variables: (12/151)
    all_times          (time) float32 480B dask.array<chunksize=(120,), meta=np.ndarray>
    areap_c            (time, psi_norm) float32 31kB dask.array<chunksize=(120, 65), meta=np.ndarray>
    betan              (time) float32 480B dask.array<chunksize=(120,), meta=np.ndarray>
    betap              (time) float32 480B dask.array<chunksize=(120,), meta=np.ndarray>
    betapd             (time) float32 480B dask.array<chunksize=(120,), meta=np.ndarray>
    betat              (time) float32 480B dask.array<chunksize=(120,), meta=np.ndarray>
    ...                 ...
    wpol               (time) float32 480B dask.array<chunksize=(120,), meta=np.ndarray>
    xpoint1_rc         (time) float32 480B dask.array<chunksize=(120,), meta=np.ndarray>
    xpoint1_zc         (time) float32 480B dask.array<chunksize=(120,), meta=np.ndarray>
    xpoint2_rc         (time) float32 480B dask.array<chunksize=(120,), meta=np.ndarray>
    xpoint2_zc         (time) float32 480B dask.array<chunksize=(120,), meta=np.ndarray>
    zbdry              (time) float32 480B dask.array<chunksize=(120,), meta=np.ndarray>
Attributes:
    description:  Basic EFIT
    file_name:    efm0304.21
    format:       IDA3
    mds_name:     None
    name:         efm
    quality:      Not Checked
    shot_id:      30421
    signal_type:  Analysed
    source:       efm
    uda_name:     EFM
    uuid:         e75f0185-8b80-58f7-a1dd-5f7fc4659a12
    version:      0

Below we show how to load and plot the plasma current denisty and with the last closed flux surface (LCFS).

d = dataset['plasma_current_rz'].dropna(dim='time')
r = dataset['r']
z = dataset['z']


lcfs_R = dataset['lcfs_r'].sel(time=d.time)
lcfs_Z = dataset['lcfs_z'].sel(time=d.time)

R, Z = np.meshgrid(r, z)

index = 50

# Get the x-point
xpoint_r = dataset['xpoint1_rc'][index]
xpoint_z = dataset['xpoint1_zc'][index]

# Get the current centre
mag_axis_r = dataset['current_centrd_r'][index]
mag_axis_z = dataset['current_centrd_z'][index]

# Get the last closed flux surface (LCFS)
lcfs_r = lcfs_R[index].values
lcfs_r = lcfs_r[~np.isnan(lcfs_r)]
lcfs_z = lcfs_Z[index].values
lcfs_z = lcfs_z[~np.isnan(lcfs_z)]

fig, ax = plt.subplots()
ax.contourf(R, Z, d[index], cmap='magma', levels=20, label='Plasma Current')
ax.plot(lcfs_r, lcfs_z, c='red', linestyle='--', label='LCFS')
ax.scatter(xpoint_r, xpoint_z, marker='x', color='green', label='X Point')
ax.scatter(mag_axis_r, mag_axis_z, marker='o', color='purple', label='Current Centre')

plt.title(f'EFIT Plasma Current & LCFS for Shot {d.attrs["shot_id"]}')
plt.ylabel('Z (m)')
plt.xlabel('R (m)')
plt.legend()

/var/folders/xr/yr8z575s52b4tbg3fj65qwx00000gp/T/ipykernel_17185/3855999503.py:28: UserWarning: The following kwargs were not used by contour: 'label'
  ax.contourf(R, Z, d[index], cmap='magma', levels=20, label='Plasma Current')

<matplotlib.legend.Legend at 0x36784d610>
_images/c21ab34535a30eeb9003428effe692e7f3b745b8366da6f0020ceca5a47e821c.png

Mirnov Coils#

Mirnov coils are primarily used to measure magnetic fluctuations in the plasma. These fluctuations can provide important information about various plasma instabilities.

They are particularly useful for studying magnetohydrodynamic (MHD) phenomena. MHD activity includes various modes of instabilities, such as kink modes and tearing modes, which can affect plasma confinement and stability.

dataset = xr.open_zarr("https://s3.echo.stfc.ac.uk/mast/level1/shots/29790.zarr", group='xmo')
dataset

<xarray.Dataset> Size: 101MB
Dimensions:                        (dim_0: 16, dim_1: 2, time: 1400000)
Coordinates:
  * dim_0                          (dim_0) int32 64B 0 1 2 3 4 ... 12 13 14 15
  * dim_1                          (dim_1) int32 8B 0 1
  * time                           (time) float64 11MB -0.1 -0.1 ... 0.6 0.6
Data variables: (12/18)
    devices_d3_acq216_025_channel  (dim_0) int32 64B dask.array<chunksize=(16,), meta=np.ndarray>
    devices_d3_acq216_025_range    (dim_0, dim_1) float32 128B dask.array<chunksize=(16, 2), meta=np.ndarray>
    devices_limit                  (dim_0) float64 128B dask.array<chunksize=(16,), meta=np.ndarray>
    omaha_1lz                      (time) float32 6MB dask.array<chunksize=(87500,), meta=np.ndarray>
    omaha_2lt                      (time) float32 6MB dask.array<chunksize=(87500,), meta=np.ndarray>
    omaha_2lz                      (time) float32 6MB dask.array<chunksize=(87500,), meta=np.ndarray>
    ...                             ...
    omaha_5lz                      (time) float32 6MB dask.array<chunksize=(87500,), meta=np.ndarray>
    omaha_5ur                      (time) float32 6MB dask.array<chunksize=(87500,), meta=np.ndarray>
    omaha_5ut                      (time) float32 6MB dask.array<chunksize=(87500,), meta=np.ndarray>
    omaha_5uz                      (time) float32 6MB dask.array<chunksize=(87500,), meta=np.ndarray>
    omaha_6lz                      (time) float32 6MB dask.array<chunksize=(87500,), meta=np.ndarray>
    time1                          (time) float64 11MB dask.array<chunksize=(87500,), meta=np.ndarray>
Attributes:
    description:  Magnetic Field Measurements: OMAHA high frequency Mirnov co...
    file_name:    xmo029790.nc
    format:       CDF
    mds_name:     None
    name:         xmo
    quality:      Not Checked
    shot_id:      29790
    signal_type:  Raw
    source:       xmo
    uda_name:     XMO
    uuid:         ff541ac3-6b1c-5373-8d33-b85fd46bc75b
    version:      -1

We can first look at the line profile for one of the Mirnov coils:

fig, ax = plt.subplots(1, 1, figsize=(10, 5))
ax.plot(dataset['time'], dataset['omaha_3lz'])
ax.grid()

ax.grid(alpha=0.3)
ax.set_ylabel('Volts (V)')
ax.set_xlabel('Time (s)')

plt.tight_layout()
_images/52ac1f59a777b27b07210f42292e53a7c2c9e8239b35f13b3df1482cd5c8fdc5.png

Looking at the spectrogram of the dataset can show us information about the MHD modes. Here we see several mode instabilities occuring before the plasma is lost.

ds = dataset['omaha_3lz']
# Parameters to limit the number of frequencies
nperseg = 2000  # Number of points per segment
nfft = 2000  # Number of FFT points

# Compute the Short-Time Fourier Transform (STFT)
sample_rate = 1/(ds.time[1] - ds.time[0])
f, t, Zxx = stft(ds, fs=int(sample_rate), nperseg=nperseg, nfft=nfft)

fig, ax = plt.subplots(figsize=(15, 5))
cax = ax.pcolormesh(t, f/1000, np.abs(Zxx), shading='nearest', cmap='jet', norm=LogNorm(vmin=1e-5))
ax.set_ylim(0, 50)
ax.set_title(f'XMO/OMAHA/3LZ - Shot {ds.attrs["shot_id"]}')
ax.set_ylabel('Frequency [Hz]')
ax.set_xlabel('Time [sec]')
plt.colorbar(cax, ax=ax)

<matplotlib.colorbar.Colorbar at 0x36506bf50>
_images/2ff70f004a26ffd224c73035ebc482ba5a88d644b06646e4c9d3416181754d23.png

Photron Camera Data#

RBA#

RBA contains the data from Photron bullet camera A.

A Photron Bullet Camera provides high-speed, high-resolution imaging of fast transient events in the plasma. Its ability to capture detailed images of plasma instabilities, turbulence, and disruptions makes it essential for understanding and controlling plasma behavior, ultimately aiding in the pursuit of sustained nuclear fusion.

dataset = xr.open_zarr(store, group='rba')
dataset

<xarray.Dataset> Size: 315MB
Dimensions:  (time: 450, height: 912, width: 768)
Coordinates:
  * time     (time) float64 4kB 0.000256 0.002256 0.004256 ... 0.6963 0.6983
Dimensions without coordinates: height, width
Data variables:
    data     (time, height, width) uint8 315MB dask.array<chunksize=(57, 114, 192), meta=np.ndarray>
Attributes: (12/48)
    CLASS:           IMAGE
    IMAGE_SUBCLASS:  IMAGE_INDEXED
    IMAGE_VERSION:   1.2
    board_temp:      0.0
    bottom:          1024
    camera:          
    ...              ...
    units:           pixels
    uuid:            f283e185-58a3-54f4-a5f4-a6fef86f9347
    vbin:            0
    version:         -1
    view:            Hl07 floor mount + FFC2 + 25mm lens + CII filter
    width:           768
plt.imshow(dataset.data[50], cmap='gray')

<matplotlib.image.AxesImage at 0x39ef01650>
_images/7645c42a101c51c744427c5fd2a58ae361b7c4cbb56ef7637013aa7885bd3196.png

RBB#

RBB contains the data from Photron bullet camera B, which is looking at the central column.

A Photron Bullet Camera provides high-speed, high-resolution imaging of fast transient events in the plasma. Its ability to capture detailed images of plasma instabilities, turbulence, and disruptions makes it essential for understanding and controlling plasma behavior, ultimately aiding in the pursuit of sustained nuclear fusion.

dataset = xr.open_zarr(store, group='rbb')
dataset

<xarray.Dataset> Size: 143MB
Dimensions:  (time: 500, height: 448, width: 640)
Coordinates:
  * time     (time) float64 4kB 1.6e-05 0.002016 0.004016 ... 0.694 0.696 0.698
Dimensions without coordinates: height, width
Data variables:
    data     (time, height, width) uint8 143MB dask.array<chunksize=(63, 112, 160), meta=np.ndarray>
Attributes: (12/48)
    CLASS:           IMAGE
    IMAGE_SUBCLASS:  IMAGE_INDEXED
    IMAGE_VERSION:   1.2
    board_temp:      0.0
    bottom:          680
    camera:          
    ...              ...
    units:           pixels
    uuid:            857f64f0-5329-5fc3-9e70-dafd4a69d4e7
    vbin:            0
    version:         -1
    view:            photron HM10 + Dalpha filter
    width:           640
plt.imshow(dataset.data[50], cmap='gray')

<matplotlib.image.AxesImage at 0x365711650>
_images/79a4baaf48a1395ee0eaa88b01d1e38c2a4839303f6779bc4b444878083afba9.png