{ "cells": [ { "cell_type": "code", "execution_count": 1, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "mtJKRsIvKnbl", "outputId": "6d41ccd8-7739-4215-fcf3-f2d20b3cfad0", "tags": [ "hide-output" ] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Requirement already satisfied: xarray in /usr/local/lib/python3.10/dist-packages (2023.7.0)\n", "Requirement already satisfied: zarr in /usr/local/lib/python3.10/dist-packages (2.18.2)\n", "Requirement already satisfied: numpy>=1.21 in /usr/local/lib/python3.10/dist-packages (from xarray) (1.25.2)\n", "Requirement already satisfied: pandas>=1.4 in /usr/local/lib/python3.10/dist-packages (from xarray) (2.0.3)\n", "Requirement already satisfied: packaging>=21.3 in /usr/local/lib/python3.10/dist-packages (from xarray) (24.1)\n", "Requirement already satisfied: asciitree in /usr/local/lib/python3.10/dist-packages (from zarr) (0.3.3)\n", "Requirement already satisfied: numcodecs>=0.10.0 in /usr/local/lib/python3.10/dist-packages (from zarr) (0.12.1)\n", "Requirement already satisfied: fasteners in /usr/local/lib/python3.10/dist-packages (from zarr) (0.19)\n", "Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.10/dist-packages (from pandas>=1.4->xarray) (2.8.2)\n", "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas>=1.4->xarray) (2023.4)\n", "Requirement already satisfied: tzdata>=2022.1 in /usr/local/lib/python3.10/dist-packages (from pandas>=1.4->xarray) (2024.1)\n", "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.2->pandas>=1.4->xarray) (1.16.0)\n" ] } ], "source": [ "!pip install xarray zarr" ] }, { "cell_type": "markdown", "metadata": { "id": "MBTnYN9sjM8N" }, "source": [ "### Bulk Download\n", "\n", "This notebook shows how to perform bulk downloads with a S3 command line tool. This is useful if you want to have local access to a big subset of the data or event download the whole archive!\n", "\n", "We can download data in bulk using any command line for that supports the S3 protocol. We recommend using the [s5cmd](https://github.com/peak/s5cmd) tool.\n", "\n", "Below we're going to install `s5cmd` and then use it to download thompson scattering data from one shot.\n", "\n", "First we install `s5cmd`:" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "yn9W1Ch1hmNX", "outputId": "bf27c5f6-4203-4300-929f-038d2a5aca79", "tags": [ "hide-output" ] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "CHANGELOG.md\n", "LICENSE\n", "README.md\n", "s5cmd\n", "NAME:\n", " s5cmd - Blazing fast S3 and local filesystem execution tool\n", "\n", "USAGE:\n", " s5cmd [global options] command [command options] [arguments...]\n", "\n", "COMMANDS:\n", " ls list buckets and objects\n", " cp copy objects\n", " rm remove objects\n", " mv move/rename objects\n", " mb make bucket\n", " rb remove bucket\n", " select run SQL queries on objects\n", " du show object size usage\n", " cat print remote object content\n", " pipe stream to remote from stdin\n", " run run commands in batch\n", " sync sync objects\n", " version print version\n", " bucket-version configure bucket versioning\n", " presign print remote object presign url\n", " help, h Shows a list of commands or help for one command\n", "\n", "GLOBAL OPTIONS:\n", " --credentials-file value use the specified credentials file instead of the default credentials file\n", " --dry-run fake run; show what commands will be executed without actually executing them (default: false)\n", " --endpoint-url value override default S3 host for custom services [$S3_ENDPOINT_URL]\n", " --help, -h show help (default: false)\n", " --install-completion get completion installation instructions for your shell (only available for bash, pwsh, and zsh) (default: false)\n", " --json enable JSON formatted output (default: false)\n", " --log value log level: (trace, debug, info, error) (default: info)\n", " --no-sign-request do not sign requests: credentials will not be loaded if --no-sign-request is provided (default: false)\n", " --no-verify-ssl disable SSL certificate verification (default: false)\n", " --numworkers value number of workers execute operation on each object (default: 256)\n", " --profile value use the specified profile from the credentials file\n", " --request-payer value who pays for request (access requester pays buckets)\n", " --retry-count value, -r value number of times that a request will be retried for failures (default: 10)\n", " --stat collect statistics of program execution and display it at the end (default: false)\n", " --use-list-objects-v1 use ListObjectsV1 API for services that don't support ListObjectsV2 (default: false)\n", " \n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "--2024-06-25 11:33:12-- https://github.com/peak/s5cmd/releases/download/v2.2.2/s5cmd_2.2.2_Linux-64bit.tar.gz\n", "Resolving github.com (github.com)... 20.27.177.113\n", "Connecting to github.com (github.com)|20.27.177.113|:443... connected.\n", "HTTP request sent, awaiting response... 302 Found\n", "Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/73909333/e095ae85-9acf-4dcc-b744-128b3311849c?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20240625%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240625T113313Z&X-Amz-Expires=300&X-Amz-Signature=2396a8ebe3d4f262b67811c8131a3e5b8561d7cfdb6679e0ec7a107e98c6fcce&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=73909333&response-content-disposition=attachment%3B%20filename%3Ds5cmd_2.2.2_Linux-64bit.tar.gz&response-content-type=application%2Foctet-stream [following]\n", "--2024-06-25 11:33:13-- https://objects.githubusercontent.com/github-production-release-asset-2e65be/73909333/e095ae85-9acf-4dcc-b744-128b3311849c?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20240625%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240625T113313Z&X-Amz-Expires=300&X-Amz-Signature=2396a8ebe3d4f262b67811c8131a3e5b8561d7cfdb6679e0ec7a107e98c6fcce&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=73909333&response-content-disposition=attachment%3B%20filename%3Ds5cmd_2.2.2_Linux-64bit.tar.gz&response-content-type=application%2Foctet-stream\n", "Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.110.133, 185.199.111.133, 185.199.109.133, ...\n", "Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.110.133|:443... connected.\n", "HTTP request sent, awaiting response... 200 OK\n", "Length: 4821452 (4.6M) [application/octet-stream]\n", "Saving to: ‘s5cmd_2.2.2_Linux-64bit.tar.gz.1’\n", "\n", " 0K .......... .......... .......... .......... .......... 1% 702K 7s\n", " 50K .......... .......... .......... .......... .......... 2% 4.27M 4s\n", " 100K .......... .......... .......... .......... .......... 3% 1.92M 3s\n", " 150K .......... .......... .......... .......... .......... 4% 5.79M 3s\n", " 200K .......... .......... .......... .......... .......... 5% 8.23M 2s\n", " 250K .......... .......... .......... .......... .......... 6% 9.17M 2s\n", " 300K .......... .......... .......... .......... .......... 7% 2.75M 2s\n", " 350K .......... .......... .......... .......... .......... 8% 15.8M 2s\n", " 400K .......... .......... .......... .......... .......... 9% 10.2M 1s\n", " 450K .......... .......... .......... .......... .......... 10% 14.0M 1s\n", " 500K .......... .......... .......... .......... .......... 11% 18.1M 1s\n", " 550K .......... .......... .......... .......... .......... 12% 15.4M 1s\n", " 600K .......... .......... .......... .......... .......... 13% 19.6M 1s\n", " 650K .......... .......... .......... .......... .......... 14% 3.22M 1s\n", " 700K .......... .......... .......... .......... .......... 15% 36.0M 1s\n", " 750K .......... .......... .......... .......... .......... 16% 21.4M 1s\n", " 800K .......... .......... .......... .......... .......... 18% 19.7M 1s\n", " 850K .......... .......... .......... .......... .......... 19% 22.1M 1s\n", " 900K .......... .......... .......... .......... .......... 20% 32.7M 1s\n", " 950K .......... .......... .......... .......... .......... 21% 19.1M 1s\n", " 1000K .......... .......... .......... .......... .......... 22% 38.8M 1s\n", " 1050K .......... .......... .......... .......... .......... 23% 54.7M 1s\n", " 1100K .......... .......... .......... .......... .......... 24% 31.5M 1s\n", " 1150K .......... .......... .......... .......... .......... 25% 21.9M 1s\n", " 1200K .......... .......... .......... .......... .......... 26% 99.1M 1s\n", " 1250K .......... .......... .......... .......... .......... 27% 30.4M 1s\n", " 1300K .......... .......... .......... .......... .......... 28% 3.59M 1s\n", " 1350K .......... .......... .......... .......... .......... 29% 33.6M 1s\n", " 1400K .......... .......... .......... .......... .......... 30% 44.2M 0s\n", " 1450K .......... .......... .......... .......... .......... 31% 51.4M 0s\n", " 1500K .......... .......... .......... .......... .......... 32% 60.1M 0s\n", " 1550K .......... .......... .......... .......... .......... 33% 36.0M 0s\n", " 1600K .......... .......... .......... .......... .......... 35% 38.4M 0s\n", " 1650K .......... .......... .......... .......... .......... 36% 45.8M 0s\n", " 1700K .......... .......... .......... .......... .......... 37% 61.5M 0s\n", " 1750K .......... .......... .......... .......... .......... 38% 45.2M 0s\n", " 1800K .......... .......... .......... .......... .......... 39% 50.5M 0s\n", " 1850K .......... .......... .......... .......... .......... 40% 59.1M 0s\n", " 1900K .......... .......... .......... .......... .......... 41% 64.6M 0s\n", " 1950K .......... .......... .......... .......... .......... 42% 26.9M 0s\n", " 2000K .......... .......... .......... .......... .......... 43% 186M 0s\n", " 2050K .......... .......... .......... .......... .......... 44% 46.6M 0s\n", " 2100K .......... .......... .......... .......... .......... 45% 54.0M 0s\n", " 2150K .......... .......... .......... .......... .......... 46% 80.2M 0s\n", " 2200K .......... .......... .......... .......... .......... 47% 238M 0s\n", " 2250K .......... .......... .......... .......... .......... 48% 42.6M 0s\n", " 2300K .......... .......... .......... .......... .......... 49% 53.5M 0s\n", " 2350K .......... .......... .......... .......... .......... 50% 49.1M 0s\n", " 2400K .......... .......... .......... .......... .......... 52% 80.2M 0s\n", " 2450K .......... .......... .......... .......... .......... 53% 261M 0s\n", " 2500K .......... .......... .......... .......... .......... 54% 64.0M 0s\n", " 2550K .......... .......... .......... .......... .......... 55% 54.0M 0s\n", " 2600K .......... .......... .......... .......... .......... 56% 47.3M 0s\n", " 2650K .......... .......... .......... .......... .......... 57% 3.99M 0s\n", " 2700K .......... .......... .......... .......... .......... 58% 247M 0s\n", " 2750K .......... .......... .......... .......... .......... 59% 44.3M 0s\n", " 2800K .......... .......... .......... .......... .......... 60% 116M 0s\n", " 2850K .......... .......... .......... .......... .......... 61% 276M 0s\n", " 2900K .......... .......... .......... .......... .......... 62% 39.0M 0s\n", " 2950K .......... .......... .......... .......... .......... 63% 48.8M 0s\n", " 3000K .......... .......... .......... .......... .......... 64% 38.3M 0s\n", " 3050K .......... .......... .......... .......... .......... 65% 42.1M 0s\n", " 3100K .......... .......... .......... .......... .......... 66% 55.6M 0s\n", " 3150K .......... .......... .......... .......... .......... 67% 62.5M 0s\n", " 3200K .......... .......... .......... .......... .......... 69% 125M 0s\n", " 3250K .......... .......... .......... .......... .......... 70% 214M 0s\n", " 3300K .......... .......... .......... .......... .......... 71% 181M 0s\n", " 3350K .......... .......... .......... .......... .......... 72% 121M 0s\n", " 3400K .......... .......... .......... .......... .......... 73% 45.4M 0s\n", " 3450K .......... .......... .......... .......... .......... 74% 50.4M 0s\n", " 3500K .......... .......... .......... .......... .......... 75% 56.2M 0s\n", " 3550K .......... .......... .......... .......... .......... 76% 41.4M 0s\n", " 3600K .......... .......... .......... .......... .......... 77% 38.3M 0s\n", " 3650K .......... .......... .......... .......... .......... 78% 44.0M 0s\n", " 3700K .......... .......... .......... .......... .......... 79% 246M 0s\n", " 3750K .......... .......... .......... .......... .......... 80% 227M 0s\n", " 3800K .......... .......... .......... .......... .......... 81% 177M 0s\n", " 3850K .......... .......... .......... .......... .......... 82% 263M 0s\n", " 3900K .......... .......... .......... .......... .......... 83% 203M 0s\n", " 3950K .......... .......... .......... .......... .......... 84% 196M 0s\n", " 4000K .......... .......... .......... .......... .......... 86% 175M 0s\n", " 4050K .......... .......... .......... .......... .......... 87% 235M 0s\n", " 4100K .......... .......... .......... .......... .......... 88% 208M 0s\n", " 4150K .......... .......... .......... .......... .......... 89% 232M 0s\n", " 4200K .......... .......... .......... .......... .......... 90% 206M 0s\n", " 4250K .......... .......... .......... .......... .......... 91% 260M 0s\n", " 4300K .......... .......... .......... .......... .......... 92% 227M 0s\n", " 4350K .......... .......... .......... .......... .......... 93% 243M 0s\n", " 4400K .......... .......... .......... .......... .......... 94% 101M 0s\n", " 4450K .......... .......... .......... .......... .......... 95% 54.7M 0s\n", " 4500K .......... .......... .......... .......... .......... 96% 51.1M 0s\n", " 4550K .......... .......... .......... .......... .......... 97% 54.0M 0s\n", " 4600K .......... .......... .......... .......... .......... 98% 47.0M 0s\n", " 4650K .......... .......... .......... .......... .......... 99% 51.3M 0s\n", " 4700K ........ 100% 34.8M=0.3s\n", "\n", "2024-06-25 11:33:14 (16.7 MB/s) - ‘s5cmd_2.2.2_Linux-64bit.tar.gz.1’ saved [4821452/4821452]\n", "\n" ] } ], "source": [ "%%bash\n", "\n", "wget https://github.com/peak/s5cmd/releases/download/v2.2.2/s5cmd_2.2.2_Linux-64bit.tar.gz\n", "tar -xvzf s5cmd_2.2.2_Linux-64bit.tar.gz\n", "./s5cmd --help" ] }, { "cell_type": "markdown", "metadata": { "id": "NSKjNdNljqhl" }, "source": [ "Now we can download data using the `cp` command.\n", "\n", "In this example, we are going to transfer the thompson scattering data for shot `30420` locally.\n", "\n", "We need to set the endpoint of where the bucket is hosted (for now: `https://s3.echo.stfc.ac.uk`) and we need to set `--no-sign-request` for annonymous access." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "_CqfL5dfiFh0", "outputId": "290bc0a1-fdc2-490b-ec00-32e146f98e9f" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "cp s3://mast/level1/shots/30420.zarr/ayc/.zgroup 30420.zarr/ayc/.zgroup\n", "cp s3://mast/level1/shots/30420.zarr/ayc/.zattrs 30420.zarr/ayc/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/version_poly/.zmetadata 30420.zarr/ayc/version_poly/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/laser_energy/.zattrs 30420.zarr/ayc/laser_energy/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/instrument_dr/.zattrs 30420.zarr/ayc/instrument_dr/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/interferometer_corr/0 30420.zarr/ayc/interferometer_corr/0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/spectra_error/.zattrs 30420.zarr/ayc/spectra_error/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/ne_error/.zattrs 30420.zarr/ayc/ne_error/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/spectra/.zmetadata 30420.zarr/ayc/spectra/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/segment_number/.zattrs 30420.zarr/ayc/segment_number/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/scat_length/.zmetadata 30420.zarr/ayc/scat_length/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/ne_core/0 30420.zarr/ayc/ne_core/0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/chi2/.zattrs 30420.zarr/ayc/chi2/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/te_error/.zmetadata 30420.zarr/ayc/te_error/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/version_raman/.zmetadata 30420.zarr/ayc/version_raman/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/spectra_stray_light/.zmetadata 30420.zarr/ayc/spectra_stray_light/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/te_core/.zattrs 30420.zarr/ayc/te_core/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/acqiris_time/.zarray 30420.zarr/ayc/acqiris_time/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/r_error/.zattrs 30420.zarr/ayc/r_error/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/scat_length/0 30420.zarr/ayc/scat_length/0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/te_error/.zarray 30420.zarr/ayc/te_error/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/arb/.zattrs 30420.zarr/ayc/arb/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/acqiris_time/0 30420.zarr/ayc/acqiris_time/0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/angle/.zattrs 30420.zarr/ayc/angle/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/aspectra/.zattrs 30420.zarr/ayc/aspectra/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/aspectra/1.0.0 30420.zarr/ayc/aspectra/1.0.0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/aspectra/.zmetadata 30420.zarr/ayc/aspectra/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/chi2/0.0 30420.zarr/ayc/chi2/0.0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/arb/.zmetadata 30420.zarr/ayc/arb/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/aspectra/.zarray 30420.zarr/ayc/aspectra/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/radialindex/.zarray 30420.zarr/ayc/radialindex/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/time_/.zarray 30420.zarr/ayc/time_/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/ne_core/.zmetadata 30420.zarr/ayc/ne_core/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/yag_nelint/.zarray 30420.zarr/ayc/yag_nelint/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/laser_track/.zarray 30420.zarr/ayc/laser_track/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/status/.zmetadata 30420.zarr/ayc/status/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/spectra_error/1.0.0 30420.zarr/ayc/spectra_error/1.0.0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/passnumber/.zattrs 30420.zarr/ayc/passnumber/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/passnumber/.zarray 30420.zarr/ayc/passnumber/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/.zmetadata 30420.zarr/ayc/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/segment_number/.zmetadata 30420.zarr/ayc/segment_number/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/radialindex/.zattrs 30420.zarr/ayc/radialindex/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/te_error/.zattrs 30420.zarr/ayc/te_error/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/spectral_index/.zattrs 30420.zarr/ayc/spectral_index/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/spectral_index/.zmetadata 30420.zarr/ayc/spectral_index/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/radialindex/0 30420.zarr/ayc/radialindex/0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/spectra_stray_light/.zarray 30420.zarr/ayc/spectra_stray_light/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/interferometer_corr/.zattrs 30420.zarr/ayc/interferometer_corr/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/te_error/0.0 30420.zarr/ayc/te_error/0.0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/angle/0 30420.zarr/ayc/angle/0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/ne_core/.zarray 30420.zarr/ayc/ne_core/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/laser_energy/0 30420.zarr/ayc/laser_energy/0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/te_core/0 30420.zarr/ayc/te_core/0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/spectral_index/0 30420.zarr/ayc/spectral_index/0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/poisson_error/.zarray 30420.zarr/ayc/poisson_error/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/instrument_dr/0.0 30420.zarr/ayc/instrument_dr/0.0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/plasmalight_error/.zmetadata 30420.zarr/ayc/plasmalight_error/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/interferometer_corr/.zmetadata 30420.zarr/ayc/interferometer_corr/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/polyname/.zmetadata 30420.zarr/ayc/polyname/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/version_poly/.zarray 30420.zarr/ayc/version_poly/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/radialindex/.zmetadata 30420.zarr/ayc/radialindex/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/xyc_time/0 30420.zarr/ayc/xyc_time/0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/xyc_time/.zarray 30420.zarr/ayc/xyc_time/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/polyname/.zattrs 30420.zarr/ayc/polyname/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/pe/.zmetadata 30420.zarr/ayc/pe/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/pulse/.zattrs 30420.zarr/ayc/pulse/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/version_fibre/.zattrs 30420.zarr/ayc/version_fibre/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/r_core/.zattrs 30420.zarr/ayc/r_core/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/spectralindex/.zmetadata 30420.zarr/ayc/spectralindex/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/time_/.zattrs 30420.zarr/ayc/time_/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/angle/.zarray 30420.zarr/ayc/angle/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/instrument_dr/.zarray 30420.zarr/ayc/instrument_dr/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/poisson_error/0.0.0 30420.zarr/ayc/poisson_error/0.0.0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/spectra_error/0.0.0 30420.zarr/ayc/spectra_error/0.0.0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/poisson_error/1.0.0 30420.zarr/ayc/poisson_error/1.0.0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/xyc_time/.zattrs 30420.zarr/ayc/xyc_time/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/chi2/.zmetadata 30420.zarr/ayc/chi2/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/covariance_ne_te/.zattrs 30420.zarr/ayc/covariance_ne_te/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/te_core/.zmetadata 30420.zarr/ayc/te_core/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/time_/.zmetadata 30420.zarr/ayc/time_/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/r/.zmetadata 30420.zarr/ayc/r/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/spectra_error/.zarray 30420.zarr/ayc/spectra_error/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/r_core/.zarray 30420.zarr/ayc/r_core/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/plasmalight_error/1.0.0 30420.zarr/ayc/plasmalight_error/1.0.0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/laser_track/0 30420.zarr/ayc/laser_track/0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/plasmalight_error/.zarray 30420.zarr/ayc/plasmalight_error/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/poisson_error/.zmetadata 30420.zarr/ayc/poisson_error/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/yag_nelint/0 30420.zarr/ayc/yag_nelint/0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/version_fibre/0 30420.zarr/ayc/version_fibre/0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/r/.zattrs 30420.zarr/ayc/r/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/spectra/.zattrs 30420.zarr/ayc/spectra/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/radial_index/.zmetadata 30420.zarr/ayc/radial_index/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/ne/.zmetadata 30420.zarr/ayc/ne/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/scat_length/.zattrs 30420.zarr/ayc/scat_length/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/spectra_stray_light/1.0.0 30420.zarr/ayc/spectra_stray_light/1.0.0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/acqiris_time/.zattrs 30420.zarr/ayc/acqiris_time/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/laser_track/.zmetadata 30420.zarr/ayc/laser_track/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/angle/.zmetadata 30420.zarr/ayc/angle/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/xyc_time/.zmetadata 30420.zarr/ayc/xyc_time/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/laser_track/.zattrs 30420.zarr/ayc/laser_track/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/spectral_index/.zarray 30420.zarr/ayc/spectral_index/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/radial_index/.zarray 30420.zarr/ayc/radial_index/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/time_/0 30420.zarr/ayc/time_/0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/spectralindex/.zattrs 30420.zarr/ayc/spectralindex/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/status/0 30420.zarr/ayc/status/0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/instrument_dr/.zmetadata 30420.zarr/ayc/instrument_dr/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/interferometer_corr/.zarray 30420.zarr/ayc/interferometer_corr/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/radial_index/0 30420.zarr/ayc/radial_index/0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/spectra_error/.zmetadata 30420.zarr/ayc/spectra_error/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/passnumber/0 30420.zarr/ayc/passnumber/0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/version_fibre/.zarray 30420.zarr/ayc/version_fibre/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/spectra_stray_light/.zattrs 30420.zarr/ayc/spectra_stray_light/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/pe/.zattrs 30420.zarr/ayc/pe/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/r_core/0 30420.zarr/ayc/r_core/0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/r_error/.zmetadata 30420.zarr/ayc/r_error/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/yag_nelint/.zattrs 30420.zarr/ayc/yag_nelint/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/segment_number/.zarray 30420.zarr/ayc/segment_number/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/pulse/.zmetadata 30420.zarr/ayc/pulse/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/radial_index/.zattrs 30420.zarr/ayc/radial_index/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/r_core/.zmetadata 30420.zarr/ayc/r_core/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/covariance_ne_te/.zarray 30420.zarr/ayc/covariance_ne_te/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/te/.zmetadata 30420.zarr/ayc/te/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/pe/0.0 30420.zarr/ayc/pe/0.0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/laser_energy/.zarray 30420.zarr/ayc/laser_energy/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/t/.zarray 30420.zarr/ayc/t/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/time/.zarray 30420.zarr/ayc/time/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/pulse/0 30420.zarr/ayc/pulse/0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/version_raman/0 30420.zarr/ayc/version_raman/0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/ne_error/0.0 30420.zarr/ayc/ne_error/0.0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/spectra/1.0.0 30420.zarr/ayc/spectra/1.0.0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/r/.zarray 30420.zarr/ayc/r/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/polyname/0 30420.zarr/ayc/polyname/0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/pe_error/.zattrs 30420.zarr/ayc/pe_error/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/te/0.0 30420.zarr/ayc/te/0.0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/scat_length/.zarray 30420.zarr/ayc/scat_length/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/pulse/.zarray 30420.zarr/ayc/pulse/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/pe/.zarray 30420.zarr/ayc/pe/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/spectralindex/.zarray 30420.zarr/ayc/spectralindex/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/t/0 30420.zarr/ayc/t/0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/version_poly/.zattrs 30420.zarr/ayc/version_poly/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/ne_error/.zmetadata 30420.zarr/ayc/ne_error/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/laser_energy/.zmetadata 30420.zarr/ayc/laser_energy/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/r_error/0.0 30420.zarr/ayc/r_error/0.0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/version_fibre/.zmetadata 30420.zarr/ayc/version_fibre/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/version_poly/0 30420.zarr/ayc/version_poly/0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/arb/0 30420.zarr/ayc/arb/0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/t/.zmetadata 30420.zarr/ayc/t/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/version_raman/.zarray 30420.zarr/ayc/version_raman/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/time/.zattrs 30420.zarr/ayc/time/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/te_core/.zarray 30420.zarr/ayc/te_core/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/time/0 30420.zarr/ayc/time/0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/polyname/.zarray 30420.zarr/ayc/polyname/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/poisson_error/.zattrs 30420.zarr/ayc/poisson_error/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/status/.zattrs 30420.zarr/ayc/status/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/acqiris_time/.zmetadata 30420.zarr/ayc/acqiris_time/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/yag_nelint/.zmetadata 30420.zarr/ayc/yag_nelint/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/pe_error/.zarray 30420.zarr/ayc/pe_error/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/ne/.zarray 30420.zarr/ayc/ne/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/status/.zarray 30420.zarr/ayc/status/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/segment_number/0 30420.zarr/ayc/segment_number/0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/te/.zarray 30420.zarr/ayc/te/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/version_raman/.zattrs 30420.zarr/ayc/version_raman/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/r/0.0 30420.zarr/ayc/r/0.0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/plasmalight_error/.zattrs 30420.zarr/ayc/plasmalight_error/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/covariance_ne_te/.zmetadata 30420.zarr/ayc/covariance_ne_te/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/te/.zattrs 30420.zarr/ayc/te/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/pe_error/.zmetadata 30420.zarr/ayc/pe_error/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/spectralindex/0 30420.zarr/ayc/spectralindex/0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/chi2/.zarray 30420.zarr/ayc/chi2/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/r_error/.zarray 30420.zarr/ayc/r_error/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/ne_error/.zarray 30420.zarr/ayc/ne_error/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/t/.zattrs 30420.zarr/ayc/t/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/time/.zmetadata 30420.zarr/ayc/time/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/spectra_stray_light/0.0.0 30420.zarr/ayc/spectra_stray_light/0.0.0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/pe_error/0.0 30420.zarr/ayc/pe_error/0.0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/covariance_ne_te/0.0 30420.zarr/ayc/covariance_ne_te/0.0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/ne_core/.zattrs 30420.zarr/ayc/ne_core/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/passnumber/.zmetadata 30420.zarr/ayc/passnumber/.zmetadata\n", "cp s3://mast/level1/shots/30420.zarr/ayc/arb/.zarray 30420.zarr/ayc/arb/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/ne/0.0 30420.zarr/ayc/ne/0.0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/ne/.zattrs 30420.zarr/ayc/ne/.zattrs\n", "cp s3://mast/level1/shots/30420.zarr/ayc/spectra/.zarray 30420.zarr/ayc/spectra/.zarray\n", "cp s3://mast/level1/shots/30420.zarr/ayc/aspectra/0.0.0 30420.zarr/ayc/aspectra/0.0.0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/spectra/0.0.0 30420.zarr/ayc/spectra/0.0.0\n", "cp s3://mast/level1/shots/30420.zarr/ayc/plasmalight_error/0.0.0 30420.zarr/ayc/plasmalight_error/0.0.0\n" ] } ], "source": [ "%%bash\n", "./s5cmd --no-sign-request --endpoint-url https://s3.echo.stfc.ac.uk cp s3://mast/level1/shots/30420.zarr/ayc/* ./30420.zarr/ayc" ] }, { "cell_type": "markdown", "metadata": { "id": "NhghzZ99jzip" }, "source": [ "Finally, we can open the file locally:" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 457 }, "id": "udo56w_3jFp_", "outputId": "911b525e-40eb-497c-b6ee-caf26b4e4f09" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ ":2: RuntimeWarning: Failed to open Zarr store with consolidated metadata, but successfully read with non-consolidated metadata. This is typically much slower for opening a dataset. To silence this warning, consider:\n", "1. Consolidating metadata in this existing store with zarr.consolidate_metadata().\n", "2. Explicitly setting consolidated=False, to avoid trying to read consolidate metadata, or\n", "3. Explicitly setting consolidated=True, to raise an error in this case instead of falling back to try reading non-consolidated metadata.\n", " xr.open_zarr('30420.zarr', group='ayc')\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
<xarray.Dataset>\n",
              "Dimensions:              (time: 146, arb: 130, radial_index: 130,\n",
              "                          spectral_index: 4, radialindex: 130,\n",
              "                          spectralindex: 4, t: 146)\n",
              "Coordinates:\n",
              "  * arb                  (arb) float32 0.0 1.0 2.0 3.0 ... 127.0 128.0 129.0\n",
              "  * radial_index         (radial_index) float32 1.0 2.0 3.0 ... 129.0 130.0\n",
              "  * radialindex          (radialindex) float32 1.0 2.0 3.0 ... 128.0 129.0 130.0\n",
              "  * spectral_index       (spectral_index) float32 1.0 2.0 3.0 4.0\n",
              "  * spectralindex        (spectralindex) float32 1.0 2.0 3.0 4.0\n",
              "  * t                    (t) float32 0.0 0.004166 0.008332 ... 70.0 71.0 72.0\n",
              "  * time                 (time) float32 0.0 0.004166 0.008332 ... 70.0 71.0 72.0\n",
              "Data variables: (12/37)\n",
              "    acqiris_time         (time) float32 dask.array<chunksize=(146,), meta=np.ndarray>\n",
              "    angle                (arb) float32 dask.array<chunksize=(130,), meta=np.ndarray>\n",
              "    aspectra             (time, radial_index, spectral_index) float32 dask.array<chunksize=(73, 130, 4), meta=np.ndarray>\n",
              "    chi2                 (time, radial_index) float32 dask.array<chunksize=(146, 130), meta=np.ndarray>\n",
              "    covariance_ne_te     (time, radial_index) float32 dask.array<chunksize=(146, 130), meta=np.ndarray>\n",
              "    instrument_dr        (time, radial_index) float32 dask.array<chunksize=(146, 130), meta=np.ndarray>\n",
              "    ...                   ...\n",
              "    time_                (time) float32 dask.array<chunksize=(146,), meta=np.ndarray>\n",
              "    version_fibre        float32 ...\n",
              "    version_poly         float32 ...\n",
              "    version_raman        float32 ...\n",
              "    xyc_time             (time) float32 dask.array<chunksize=(146,), meta=np.ndarray>\n",
              "    yag_nelint           (time) float32 dask.array<chunksize=(146,), meta=np.ndarray>\n",
              "Attributes:\n",
              "    description:  Core Thomson scattering data\n",
              "    file_name:    ayc0304.20\n",
              "    format:       IDA3\n",
              "    mds_name:     None\n",
              "    name:         ayc\n",
              "    quality:      Not Checked\n",
              "    shot_id:      30420\n",
              "    signal_type:  Analysed\n",
              "    source:       ayc\n",
              "    uda_name:     AYC\n",
              "    uuid:         8d043ece-8bf8-5af8-87e4-d2a1b01716fa\n",
              "    version:      0
" ], "text/plain": [ "\n", "Dimensions: (time: 146, arb: 130, radial_index: 130,\n", " spectral_index: 4, radialindex: 130,\n", " spectralindex: 4, t: 146)\n", "Coordinates:\n", " * arb (arb) float32 0.0 1.0 2.0 3.0 ... 127.0 128.0 129.0\n", " * radial_index (radial_index) float32 1.0 2.0 3.0 ... 129.0 130.0\n", " * radialindex (radialindex) float32 1.0 2.0 3.0 ... 128.0 129.0 130.0\n", " * spectral_index (spectral_index) float32 1.0 2.0 3.0 4.0\n", " * spectralindex (spectralindex) float32 1.0 2.0 3.0 4.0\n", " * t (t) float32 0.0 0.004166 0.008332 ... 70.0 71.0 72.0\n", " * time (time) float32 0.0 0.004166 0.008332 ... 70.0 71.0 72.0\n", "Data variables: (12/37)\n", " acqiris_time (time) float32 dask.array\n", " angle (arb) float32 dask.array\n", " aspectra (time, radial_index, spectral_index) float32 dask.array\n", " chi2 (time, radial_index) float32 dask.array\n", " covariance_ne_te (time, radial_index) float32 dask.array\n", " instrument_dr (time, radial_index) float32 dask.array\n", " ... ...\n", " time_ (time) float32 dask.array\n", " version_fibre float32 ...\n", " version_poly float32 ...\n", " version_raman float32 ...\n", " xyc_time (time) float32 dask.array\n", " yag_nelint (time) float32 dask.array\n", "Attributes:\n", " description: Core Thomson scattering data\n", " file_name: ayc0304.20\n", " format: IDA3\n", " mds_name: None\n", " name: ayc\n", " quality: Not Checked\n", " shot_id: 30420\n", " signal_type: Analysed\n", " source: ayc\n", " uda_name: AYC\n", " uuid: 8d043ece-8bf8-5af8-87e4-d2a1b01716fa\n", " version: 0" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import xarray as xr\n", "xr.open_zarr('30420.zarr', group='ayc')" ] } ], "metadata": { "colab": { "provenance": [] }, "kernelspec": { "display_name": "mast-book", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.9" } }, "nbformat": 4, "nbformat_minor": 0 }