# NSG CLI

A command-line interface for the [Neuroscience Gateway (NSG)](https://www.nsgportal.org/) BRAIN Initiative API.

## Features

- **Secure credential storage** - Store NSG credentials in `~/.nsg/credentials.json`
- **Job management** - Submit, monitor, and download results from NSG HPC jobs
- **Parallel job fetching** - Fast job listing with concurrent API requests (up to 2x speedup)
- **Beautiful CLI** - Colored output with progress indicators
- **XML API support** - Full support for NSG's REST API (XML-based)
- **Multiple commands** - Login, list, status, submit, and download operations
- **Flexible compilation** - Feature flags for parallel vs sequential processing

## Installation

### From `crates.io`

```bash
cargo install nsg-cli
```

### Build from source

```bash
cd nsg-cli
cargo build --release
```

The binary will be at `target/release/nsg`.

### Install globally

```bash
cargo install --path .
```

This installs the `nsg` binary to `~/.cargo/bin/`.

## Performance

### Parallel Job Fetching

By default, `nsg-cli` uses parallel processing to fetch job details concurrently, significantly improving performance when listing many jobs.

**Benchmark Results (43 jobs):**
- **Parallel (default)**: ~1.75 seconds
- **Sequential**: ~3.47 seconds
- **Speedup**: ~2x faster with parallel processing

### Build Options

```bash
# Default build with parallel support (recommended)
cargo build --release

# Build without parallel support (sequential only)
cargo build --release --no-default-features

# Explicitly enable parallel support
cargo build --release --features parallel
```

### When to Use Sequential

Sequential processing may be preferable when:
- Running on systems with limited CPU resources
- API rate limiting is a concern
- You have very few jobs (< 5)

For most users, the default parallel build provides the best performance.

See [BENCHMARKING.md](BENCHMARKING.md) for detailed benchmarking instructions.

## Quick Start

### 1. Login

First, authenticate with your NSG credentials:

```bash
nsg login
```

You'll be prompted for:

- NSG Username
- NSG Password (hidden input)
- NSG Application Key

Your credentials are stored in `~/.nsg/credentials.json` with secure permissions (0600 on Unix).

**Get NSG credentials at:** https://www.nsgportal.org/

### 2. List Jobs

View all your NSG jobs with detailed information (tool, status, dates):

```bash
nsg list
```

The list command automatically fetches detailed information for each job, including:
- Tool name (e.g., PY_EXPANSE)
- Job status (COMPLETED, RUNNING, etc.)
- Submission date
- Completion date
- Failed status

For additional job messages:

```bash
nsg list --detailed
```

### 3. Check Job Status

Monitor a specific job:

```bash
nsg status <JOB_ID>
```

Examples:

```bash
nsg status NGBW-JOB-PY_EXPANSE-xxxxx
nsg status https://nsgr.sdsc.edu:8443/cipresrest/v1/job/username/NGBW-JOB-PY_EXPANSE-xxxxx
```

### 4. Submit a Job

Submit a new job to NSG:

```bash
nsg submit <ZIP_FILE> --tool <TOOL_NAME>
```

Example:

```bash
nsg submit my_analysis.zip --tool PY_EXPANSE
```

Available tools:

- `PY_EXPANSE` - Python on EXPANSE (default)
- `GPU_PY_EXPANSE` - GPU-accelerated Python
- Other NSG tools as supported

### 5. Download Results

Download results from a completed job:

```bash
nsg download <JOB_ID>
```

Specify output directory:

```bash
nsg download <JOB_ID> --output ./my_results
```

## Commands

### `nsg login`

Authenticate and save credentials.

**Options:**

- `-u, --username <USERNAME>` - NSG username (or prompt)
- `-p, --password <PASSWORD>` - NSG password (or prompt securely)
- `-a, --app-key <APP_KEY>` - NSG application key (or prompt)
- `--no-verify` - Skip connection test

**Example:**

```bash
nsg login --username myuser --app-key MY_APP_KEY
```

### `nsg list`

List all jobs for the authenticated user with detailed information (tool, status, dates).

**Note:** By default, this command fetches detailed status for each job using parallel processing for optimal performance.

**Options:**

- `--detailed` - Show job messages for each job
- `--recent <N>` - Show only the N most recent jobs (default: 20)
- `--limit <N>` - Limit number of jobs to display
- `--all` - Show all jobs (override default 20-job limit)

**Job Information Displayed:**
- Job ID
- Tool name (e.g., PY_EXPANSE)
- Status (COMPLETED, RUNNING, FAILED, etc.)
- Submission timestamp
- Completion timestamp (for terminal jobs)
- Failed status

**Examples:**

```bash
nsg list                    # Show 20 most recent jobs with details
nsg list --all              # Show all jobs with details
nsg list --recent 5         # Show 5 most recent jobs
nsg list --limit 10         # Show first 10 jobs
nsg list --detailed         # Show job messages for recent jobs
nsg list --all --detailed   # Show all jobs with messages
```

### `nsg status <JOB>`

Check status of a specific job.

**Arguments:**

- `<JOB>` - Job URL or Job ID

**Example:**

```bash
nsg status NGBW-JOB-PY_EXPANSE-xxxxx
```

### `nsg submit <ZIP_FILE>`

Submit a new job to NSG.

**Arguments:**

- `<ZIP_FILE>` - Path to ZIP file containing job data

**Options:**

- `-t, --tool <TOOL>` - NSG tool to use (default: PY_EXPANSE)
- `--no-wait` - Don't wait for job submission confirmation

**Example:**

```bash
nsg submit job_data.zip --tool PY_EXPANSE
```

### `nsg download <JOB>`

Download results from a completed job with real-time progress tracking.

**Arguments:**

- `<JOB>` - Job URL or Job ID

**Options:**

- `-o, --output <DIR>` - Output directory (default: ./nsg_results)

**Features:**

- Real-time progress bar showing download speed and ETA
- File size display in appropriate units (B, KB, MB, GB)
- Automatic file size formatting

**Example:**

```bash
nsg download NGBW-JOB-PY_EXPANSE-xxxxx --output ./results
```

## NSG Job Package Structure

When submitting jobs, NSG expects a specific ZIP structure. For Python jobs:

```
job.zip
└── modeldir/
    ├── input.py          # Main Python script (required)
    ├── data.edf          # Input data files
    ├── params.json       # Configuration
    └── ...               # Other files
```

The main script should be named `input.py` for PY_EXPANSE tool.

## Configuration

Credentials are stored in: `~/.nsg/credentials.json`

**Format:**

```json
{
  "username": "your_username",
  "password": "your_password",
  "app_key": "your_app_key"
}
```

**Security:**

- On Unix systems, the file permissions are set to `0600` (read/write for owner only)
- Never commit this file to version control
- Keep your credentials secure

## API Documentation

This CLI interfaces with the NSG REST API:

- **Base URL:** `https://nsgr.sdsc.edu:8443/cipresrest/v1`
- **Authentication:** HTTP Basic Auth + `cipres-appkey` header
- **Response format:** XML

## Development

### Project Structure

```
nsg-cli/
├── Cargo.toml
├── src/
│   ├── main.rs           # CLI entry point
│   ├── lib.rs            # Library exports
│   ├── client.rs         # NSG API client
│   ├── config.rs         # Credential management
│   ├── models.rs         # Data structures & XML parsing
│   └── commands/         # CLI commands
│       ├── mod.rs
│       ├── login.rs
│       ├── list.rs
│       ├── status.rs
│       ├── submit.rs
│       └── download.rs
└── README.md
```

### Dependencies

- **clap** - CLI argument parsing
- **reqwest** - HTTP client
- **quick-xml** - XML parsing
- **serde** - Serialization
- **colored** - Terminal colors
- **indicatif** - Progress bars
- **rpassword** - Secure password input
- **rayon** - Parallel iteration for performance
- **criterion** - Benchmarking framework (dev dependency)

### Building

```bash
cargo build                      # Debug build
cargo build --release            # Release build with parallel support (optimized)
cargo build --release --no-default-features  # Sequential build
cargo test                       # Run tests
cargo check                      # Type checking only (fast)
cargo bench                      # Run benchmarks
```

### Benchmarking

Compare parallel vs sequential performance:

```bash
# Benchmark with parallel support (default)
cargo bench --features parallel

# Benchmark without parallel support
cargo bench --no-default-features

# View benchmark reports
open target/criterion/report/index.html
```

For detailed benchmarking instructions, see [BENCHMARKING.md](BENCHMARKING.md).

## Troubleshooting

### Authentication Failed

If login fails:

1. Verify credentials at https://www.nsgportal.org/
2. Check that your NSG account is active
3. Ensure application key is correct
4. Try using `--no-verify` to skip connection test and save credentials anyway

### Job Not Found

If `status` or `download` can't find a job:

1. Verify job ID is correct
2. Use `nsg list` to see all your jobs
3. Try using the full job URL instead of just the ID

### Download Failed

If results download fails:

1. Check job is in COMPLETED stage with `nsg status`
2. Verify job has results available
3. Check output directory permissions

## License

MIT License

## Related Projects

- [DDALAB](https://github.com/sdraeger/DDALAB) - Delay Differential Analysis Laboratory
- [NSG Portal](https://www.nsgportal.org/) - Neuroscience Gateway web interface

## Contributing

This tool was created as part of the DDALAB project. For issues or contributions, please use the DDALAB repository.

## Contact

For NSG-related issues: nsghelp@sdsc.edu
For DDALAB issues: https://github.com/sdraeger/DDALAB/issues
