<img align="right" width="180" height="180" src="https://raw.githubusercontent.com/zellerlab/GECCO/v0.6.2/static/gecco-square.png">

# Hi, I'm GECCO!

## 🦎 ️Overview

GECCO (Gene Cluster prediction with Conditional Random Fields) is a fast and
scalable method for identifying putative novel Biosynthetic Gene Clusters (BGCs)
in genomic and metagenomic data using Conditional Random Fields (CRFs).

[![Actions](https://img.shields.io/github/actions/workflow/status/zellerlab/GECCO/test.yml?branch=master&style=flat-square&maxAge=300)](https://github.com/zellerlab/GECCO/actions/workflows/test.yml)
[![License](https://img.shields.io/badge/license-GPLv3-blue.svg?style=flat-square&maxAge=2678400)](https://choosealicense.com/licenses/gpl-3.0/)
[![Coverage](https://img.shields.io/codecov/c/gh/zellerlab/GECCO?style=flat-square&maxAge=600)]( https://codecov.io/gh/zellerlab/GECCO/)
[![Docs](https://img.shields.io/badge/docs-gecco.embl.de-green.svg?maxAge=2678400&style=flat-square)](https://gecco.embl.de)
[![Source](https://img.shields.io/badge/source-GitHub-303030.svg?maxAge=2678400&style=flat-square)](https://github.com/zellerlab/GECCO/)
[![Mirror](https://img.shields.io/badge/mirror-EMBL-009f4d?style=flat-square&maxAge=2678400)](https://git.embl.de/grp-zeller/GECCO/)
[![Changelog](https://img.shields.io/badge/keep%20a-changelog-8A0707.svg?maxAge=2678400&style=flat-square)](https://github.com/zellerlab/GECCO/blob/master/CHANGELOG.md)
[![Issues](https://img.shields.io/github/issues/zellerlab/GECCO.svg?style=flat-square&maxAge=600)](https://github.com/zellerlab/GECCO/issues)
[![Preprint](https://img.shields.io/badge/preprint-bioRxiv-darkblue?style=flat-square&maxAge=2678400)](https://www.biorxiv.org/content/10.1101/2021.05.03.442509v1)
[![PyPI](https://img.shields.io/pypi/v/gecco.svg?style=flat-square&maxAge=3600)](https://pypi.python.org/pypi/gecco)
[![Bioconda](https://img.shields.io/conda/vn/bioconda/gecco?style=flat-square&maxAge=3600)](https://anaconda.org/bioconda/gecco)
[![Galaxy](https://img.shields.io/badge/Galaxy-GECCO-darkblue?style=flat-square&maxAge=3600)](https://toolshed.g2.bx.psu.edu/repository?repository_id=c29bc911b3fc5f8c)
[![Versions](https://img.shields.io/pypi/pyversions/gecco.svg?style=flat-square&maxAge=3600)](https://pypi.org/project/gecco/#files)
[![Wheel](https://img.shields.io/pypi/wheel/gecco?style=flat-square&maxAge=3600)](https://pypi.org/project/gecco/#files)


## 🔧 Installing GECCO

GECCO is implemented in [Python](https://www.python.org/), and supports [all
versions](https://endoflife.date/python) from Python 3.7. It requires
additional libraries that can be installed directly from
[PyPI](https://pypi.org), the Python Package Index.

Use [`pip`](https://pip.pypa.io/en/stable/) to install GECCO on your
machine:
```console
$ pip install gecco
```

If you'd rather use [Conda](https://conda.io), a package is available
in the [`bioconda`](https://bioconda.github.io/) channel. You can install
with:
```console
$ conda install -c bioconda gecco
```

This will install GECCO, its dependencies, and the data needed to run
predictions. This requires around 40MB of data to be downloaded, so
it could take some time depending on your Internet connection. Once done,
you will have a ``gecco`` command available in your $PATH.

*Since release `v0.10.2`, GECCO can now run on Windows! This uses
the PyHMMER `v0.12.0` experimental [MinGW-w64](https://www.mingw-w64.org/) 
build which supports Windows 10 and later. See the PyHMMER
documentation for more information about 
[Windows support](https://pyhmmer.readthedocs.io/en/stable/guide/windows.html)*.

## 🧬 Running GECCO

Once `gecco` is installed, you can run it from the terminal by giving it a
FASTA or GenBank file with the genomic sequence you want to analyze, as
well as an output directory:

```console
$ gecco run --genome some_genome.fna -o some_output_dir
```

Additional parameters of interest are:

- `--jobs`, which controls the number of threads that will be spawned by
  GECCO whenever a step can be parallelized. The default, *0*, will
  autodetect the number of CPUs on the machine using
  [`os.cpu_count`](https://docs.python.org/3/library/os.html#os.cpu_count).
- `--cds`, controlling the minimum number of consecutive genes a BGC region
  must have to be detected by GECCO. The default is *3*.
- `--threshold`, controlling the minimum probability for a gene to be
  considered part of a BGC region. Using a lower number will increase the
  number (and possibly length) of predictions, but reduce accuracy. The
  default of *0.8* was selected to optimize precision/recall on a test set
  of 364 BGCs from [MIBiG 2.0](https://mibig.secondarymetabolites.org/).
- `--cds-feature`, which can be supplied a feature name to extract genes
  if the input file already contains gene annotations instead of predicting
  genes with [Pyrodigal](https://pyrodigal.readthedocs.io). A common value
  for records downloaded from GenBank is `--cds-feature CDS`.

### Global Options

These options are available for all commands:

| Flag | Description | Default |
|------|-------------|---------|
| `-j`, `--jobs` | Number of threads for multithreading (`0` = all CPUs) | `0` |
| `-v`, `--verbose` | Increase verbosity (repeat for more, e.g. `-vv`) | — |
| `-q`, `--quiet` | Reduce or disable console output | — |
| `--no-color` | Disable console color output | — |
| `--no-markup` | Disable console markup formatting | — |

### `gecco run`

Full pipeline: gene finding, domain annotation, CRF prediction, cluster refinement, and type classification.

**Input:**

| Flag | Description | Default |
|------|-------------|---------|
| `-g`, `--genome` | Input genomic file (FASTA, GenBank, EMBL, etc.) | *required* |
| `-f`, `--format` | Biopython format string (auto-detected for common formats) | auto |

**Gene calling:**

| Flag | Description | Default |
|------|-------------|---------|
| `-M`, `--mask` | Mask unknown regions to prevent genes stretching across ambiguous nucleotides | off |
| `--cds-feature` | Extract genes from annotated features instead of de-novo prediction | — |
| `--locus-tag` | Feature qualifier for naming extracted genes | `locus_tag` |

**Domain annotation:**

| Flag | Description | Default |
|------|-------------|---------|
| `--hmm` | Path to alternative HMM file(s) (can be repeated) | embedded |
| `-e`, `--e-filter` | E-value cutoff for protein domains | — |
| `-p`, `--p-filter` | P-value cutoff for protein domains | `1e-9` |
| `--bit-cutoffs` | HMM-specific bitscore cutoffs (`noise`, `gathering`, `trusted`) | — |

**Cluster detection:**

| Flag | Description | Default |
|------|-------------|---------|
| `--model` | Path to alternative CRF model (from `gecco train`) | embedded |
| `--no-pad` | Disable padding of gene sequences | off |
| `-c`, `--cds` | Minimum coding sequences per cluster | `3` |
| `-m`, `--threshold` | Probability threshold for cluster detection | `0.8` |
| `-E`, `--edge-distance` | Minimum annotated genes separating cluster from sequence edge | `0` |
| `--trim` / `--no-trim` | Trim genes lacking domain annotations at cluster edges | on |

**Output:**

| Flag | Description | Default |
|------|-------------|---------|
| `-o`, `--output-dir` | Output directory | `.` |
| `--force-tsv` | Write TSV files even when empty | off |
| `--merge-gbk` | Single GenBank file for all clusters instead of one per cluster | off |
| `--antismash-sideload` | Write AntiSMASH v6 sideload JSON file | off |

### `gecco annotate`

Run only gene finding and domain annotation (no cluster prediction).

| Flag | Description | Default |
|------|-------------|---------|
| `-g`, `--genome` | Input genomic file | *required* |
| `-f`, `--format` | Biopython format string | auto |
| `-M`, `--mask` | Mask unknown regions | off |
| `--cds-feature` | Extract genes from annotated features | — |
| `--locus-tag` | Feature qualifier for naming genes | `locus_tag` |
| `--hmm` | Alternative HMM file(s) | embedded |
| `-e`, `--e-filter` | E-value cutoff | — |
| `-p`, `--p-filter` | P-value cutoff | `1e-9` |
| `--bit-cutoffs` | HMM bitscore cutoffs (`noise`, `gathering`, `trusted`) | — |
| `-o`, `--output-dir` | Output directory | `.` |
| `--force-tsv` | Write TSV files even when empty | off |

### `gecco predict`

Predict clusters from pre-annotated feature tables.

| Flag | Description | Default |
|------|-------------|---------|
| `-f`, `--features` | Domain annotation table (can be repeated) | *required* |
| `-g`, `--genes` | Gene coordinate table | *required* |
| `--genome` | Genomic file for sequence loading | — |
| `--format` | Biopython format string | auto |
| `-e`, `--e-filter` | E-value cutoff | — |
| `-p`, `--p-filter` | P-value cutoff | `1e-9` |
| `--model` | Alternative CRF model directory | embedded |
| `--no-pad` | Disable feature padding | off |
| `-c`, `--cds` | Minimum coding sequences per cluster | `3` |
| `-m`, `--threshold` | Probability threshold | `0.8` |
| `-E`, `--edge-distance` | Minimum genes from edge | `0` |
| `--trim` / `--no-trim` | Trim unannotated edge genes | on |
| `-o`, `--output-dir` | Output directory | `.` |
| `--force-tsv` | Write TSV files even when empty | off |
| `--merge-gbk` | Single GenBank file for all clusters | off |
| `--antismash-sideload` | Write AntiSMASH sideload JSON | off |

### `gecco train`

Train a new CRF model on pre-generated annotation tables.

| Flag | Description | Default |
|------|-------------|---------|
| `-f`, `--features` | Domain annotation table (can be repeated) | *required* |
| `-g`, `--genes` | Gene coordinate table | *required* |
| `-c`, `--clusters` | Cluster annotation table | *required* |
| `-e`, `--e-filter` | E-value cutoff | — |
| `-p`, `--p-filter` | P-value cutoff | `1e-9` |
| `--no-shuffle` | Disable data shuffling before fitting | off |
| `--seed` | Random number generator seed | `42` |
| `-W`, `--window-size` | CRF sliding window length | `5` |
| `--window-step` | CRF sliding window step | `1` |
| `--c1` | L1 regularization strength | `0.15` |
| `--c2` | L2 regularization strength | `0.15` |
| `--feature-type` | Feature extraction level (`protein` or `domain`) | `protein` |
| `--select` | Fraction of most significant features to select | all |
| `--correction` | Multiple test correction method for feature selection | — |
| `-o`, `--output-dir` | Output directory | `.` |

### `gecco cv`

Cross-validation for model evaluation. Supports K-fold and Leave-One-Type-Out.

| Flag | Description | Default |
|------|-------------|---------|
| `-f`, `--features` | Domain annotation table (can be repeated) | *required* |
| `-g`, `--genes` | Gene coordinate table | *required* |
| `-c`, `--clusters` | Cluster annotation table | *required* |
| `-e`, `--e-filter` | E-value cutoff | — |
| `-p`, `--p-filter` | P-value cutoff | `1e-9` |
| `--no-shuffle` | Disable data shuffling | off |
| `--seed` | Random number generator seed | `42` |
| `-W`, `--window-size` | CRF sliding window length | `5` |
| `--window-step` | CRF sliding window step | `1` |
| `--c1` | L1 regularization strength | `0.15` |
| `--c2` | L2 regularization strength | `0.15` |
| `--feature-type` | Feature extraction level (`protein` or `domain`) | `protein` |
| `--select` | Fraction of features to select | all |
| `--correction` | Multiple test correction method | — |
| `--loto` | Use Leave-One-Type-Out instead of K-folds | off |
| `--splits` | Number of K-fold splits | `10` |
| `-o`, `--output` | Output file path | `cv.tsv` |

### `gecco convert`

Convert output files to other formats.

**`gecco convert gbk`** — Convert GenBank files:

| Flag | Description | Default |
|------|-------------|---------|
| `-i`, `--input-dir` | Input directory | *required* |
| `-o`, `--output-dir` | Output directory | same as input |
| `-f`, `--format` | Output format: `bigslice`, `fna`, or `faa` | *required* |

**`gecco convert clusters`** — Convert cluster tables:

| Flag | Description | Default |
|------|-------------|---------|
| `-i`, `--input-dir` | Input directory | *required* |
| `-o`, `--output-dir` | Output directory | same as input |
| `-f`, `--format` | Output format: `gff` | *required* |

## 🔎 Results

GECCO will create the following files:

- `{genome}.genes.tsv`: The *genes* file, containing the genes extracted
  or predicted from the input file, and per-gene BGC probabilities
  predicted by the CRF.
- `{genome}.features.tsv`: The *features* file, containing the identified
  domains in the input sequences, in tabular format.
- `{genome}.clusters.tsv`: If any were found, a *clusters* file, containing
  the coordinates of the predicted clusters along their putative biosynthetic
  type, in tabular format.
- `{genome}_cluster_{N}.gbk`: If any were found, a GenBank file per cluster,
  containing the cluster sequence annotated with its member proteins and domains.

GECCO can also convert results to other formats that may be more convenient
depending on the downstream usage. GECCO can convert results into:

- GFF3 format so they can be loaded into a genomic viewer 
  (`gecco convert clusters --format gff`).
- GenBank files with antiSMASH-style features so they can be loaded into 
  [BiG-SLiCE](https://github.com/medema-group/bigslice) for further analysis
  (`gecco convert gbk --format bigslice`).
- FASTA files with the sequences of all the predicted BGCs (`gecco convert gbk --format fna`)
  or with the sequences of all their proteins (`gecco convert gbk --format faa`).

To get a more visual way of exploring of the predictions, you
can open the GenBank files in a genome editing software like [UGENE](http://ugene.net/).
You can otherwise load the results into an AntiSMASH report: check the
[Integrations](https://gecco.embl.de/integrations.html#antismash) page of the
documentation for a step-by-step guide. 


## 🔖 Reference

GECCO can be cited using the following preprint:

> **Accurate de novo identification of biosynthetic gene clusters with GECCO**.
> Laura M Carroll, Martin Larralde, Jonas Simon Fleck, Ruby Ponnudurai, Alessio Milanese, Elisa Cappio Barazzone, Georg Zeller.
> bioRxiv 2021.05.03.442509; [doi:10.1101/2021.05.03.442509](https://doi.org/10.1101/2021.05.03.442509)


## 💭 Feedback

### ⚠️ Issue Tracker

Found a bug ? Have an enhancement request ? Head over to the [GitHub issue
tracker](https://github.com/zellerlab/GECCO/issues) if you need to report
or ask something. If you are filing in on a bug, please include as much
information as you can about the issue, and try to recreate the same bug
in a simple, easily reproducible situation.

### 🏗️ Contributing

Contributions are more than welcome! See [`CONTRIBUTING.md`](https://github.com/zellerlab/GECCO/blob/master/CONTRIBUTING.md)
for more details.

### 🤝 Acknowledgments

We thank Maarten van Gompel ([@proycon](https://github.com/proycon)), the author of the homonymous
Gecco package (Generic Environment for Context-Aware Correction of Orthography), for allowing us 
to take ownership of the PyPI package name. The Gecco releases (up to `0.3.0`) can still 
be downloaded from the [PyPI project page](https://pypi.org/project/gecco/#history)) of the
same name.


## ⚖️ License

This software is provided under the [GNU General Public License v3.0 *or later*](https://choosealicense.com/licenses/gpl-3.0/). GECCO is developped by the [Zeller Team](https://www.embl.de/research/units/scb/zeller/index.html)
at the [European Molecular Biology Laboratory](https://www.embl.de/) in Heidelberg.

