Greetings GenBank Users,
GenBank Release 141.0 is now available via ftp from the National
Center for Biotechnology Information (NCBI):
Ftp Site Directory Contents
---------------- --------- ---------------------------------------
ftp.ncbi.nih.gov genbank GenBank Release 141.0 flatfiles
ncbi-asn1 ASN.1 data used to create Release 141.0
Uncompressed, the Release 141.0 flatfiles require approximately 131 GB
(sequence files only) or 146 GB (including the 'short directory' and
'index' files). The ASN.1 version requires approximately 115 GB. From
the release notes:
Release Date Base Pairs Entries
140 Feb 2004 37893844733 32549400
141 Apr 2004 38989342565 33676218
Close-of-data was 04/20/2004. Five working days were required to prepare
this release. In the eight week period between the close dates for GenBank
releases 140.0 and 141.0, the non-WGS portion of GenBank grew by 1,095,497,832
basepairs and by 1,126,818 sequence records. During that same period, 556,412
records were updated. Combined, this yields an average of about 28,500 new
and/or updated records per day.
Between releases 140.0 and 141.0, the WGS component of GenBank grew by
1,954,410,330 basepairs and by 923,778 sequence records.
*NOTE* Problems were encountered during release processing which
prevented the generation of the gbacc.idx and gbjou.idx 'index' files
for GenBank 141.0. Please see Section 1.3.1 of the release notes for
further details.
We would like to remind our users that GenBank mirrors are available
at ftp://genbank.sdsc.edu/pub and ftp://bio-mirror.net/biomirror/genbank.
Those who experience slow FTP transfers due to a high volume of traffic at
NCBI might realize an improvement in transfer rates from these alternate sites.
For additional release information, see the README files in either of
the directories mentioned above, and the release notes (gbrel.txt) in
the genbank directory. Sections 1.3 and 1.4 of the release notes
(Changes in Release 141.0 and Upcoming Changes) have been appended
below.
*NOTE* Section 1.4.1 discusses a very important change : the removal
of sequence length limits for all classes of GenBank sequence records,
as of June 2004. We strongly encourage all users to review this
information.
Release 141.0 data, and subsequent updates, are available now via
NCBI's Entrez and Blast services.
If you encounter problems while ftp'ing or uncompressing Release
141.0, please send email outlining your difficulties to
info at ncbi.nlm.nih.gov .
Mark Cavanaugh, Vladimir Alekseyev, Anton Butanaev, Michael Kimelman
GenBank
NCBI/NLM/NIH
1.3 Important Changes in Release 141.0
1.3.1 Two 'index' files unavailable for GenBank 141.0
An unexpected software problem prevented the generation of two of
the 'index' files which normally accompany GenBank releases:
gbacc.idx
gbjou.idx
Resolving the problem would have delayed release processing by three
days, with a concommitant delay in the indexing of recently processed
records for Entrez. So it was decided to make GenBank 141.0 available
without these index files. Our apologies for any inconvenience that this
might cause.
1.3.2 Organizational changes
The total number of sequence data files increased by 16 with this release:
- the BCT division is now comprised of 9 files (+1)
- the EST division is now comprised of 305 files (+9)
- the GSS division is now comprised of 104 files (+2)
- the PLN division is now comprised of 11 files (+1)
- the ROD division is now comprised of 12 files (+1)
- the STS division is now comprised of 4 files (+1)
- the VRT division is now comprised of 6 files (+1)
1.3.3 GSS File Header Problem
GSS sequences at GenBank are maintained in one of two different systems,
depending on their origin. One recent change to release processing involves
the parallelization of the dumps from those systems. Because the second dump
(for example) has no prior knowledge of exactly how many GSS files will be
dumped from the first, it doesn't know how to number it's own output files.
There is thus a discrepancy between the filenames and file headers for
thirteen GSS flatfiles in Release 141.0. Consider the gbgss92.seq file:
GBGSS1.SEQ Genetic Sequence Data Bank
April 15 2004
NCBI-GenBank Flat File Release 141.0
GSS Sequences (Part 1)
88249 loci, 65635323 bases, from 88249 reported sequences
Here, the filename and part number in the header is "1", though the file
has been renamed as "92" based on the files dumped from the other system.
We will work to resolve this discrepancy in future releases, but the
priority is certainly much lower than many other tasks.
1.4 Upcoming Changes
1.4.1 **Sequence Length Limitation To Be Removed In June 2004**
At the May 2003 collaborative meeting among representatives of GenBank,
EMBL, and DDBJ, it was decided that the 350 kilobase limit on the sequence
length of database records will be removed as of June 2004.
Individual, complete sequences are currently expected to be a maximum
of 350 kbp in length. One major reason for the existence of this limit is
as an aid to users of sequence analysis software, some of which might not
be capable of processing megabase-scale sequences.
However, very significant exceptions to the 350 kbp limit have existed
for several years; Phase 1 (unordered, unoriented) and Phase 2 (ordered,
oriented) high-throughput genomic sequences (HTGS) generated by efforts
such as the Human Genome Project; large dispersed eukaryotic genes with
an intron/exon structure that spans more than 350 kbp; and sequences
which result from assemblies of Whole Genome Shotgun (WGS) project data.
Given these exceptions, and the technological advances which have made
large-scale sequencing practical for an increasing number of researchers,
the collaboration has decided that the 350 kbp limit must be removed.
As of June 2004, the length of database sequences will be limited only
by the natural structures of an organism's genome. For example, a single
record might be used to represent all of human chromosome 1, which is
approximately 245 Mbp in length.
Software developers for some of the larger commercial sequence analysis
packages were recently asked what timeframe would be appropriate for this
change. Answers ranged from "immediately", to "several months", to "one year".
So the one-year timeframe was selected, to provide ample time to implement
changes which megabase-scale sequences may require.
Some sample records with very large sequences have been made available
so that developers can begin to test their software modifications:
ftp://ftp.ncbi.nih.gov/genbank/LargeSeqs
Many changes are expected after the removal of the length limit. For
example, complete bacterial genomes (typically on the order of several
megabases) will be re-assembled into single sequence records. The submission
process for such genomes will become much more streamlined, since database
staff will no longer have to split the genomes into pieces. BLAST services
will be enchanced, so that hits reported within very large sequences will
be presented in a meaningful context.
All such changes will be discussed more fully in future release notes,
the NCBI newsletter, and the GenBank newsgroup.
1.4.2 Rename of File 'Last.Release' and Deletion of /daily Subdirectories
The files named Last.Release which are located at:
ftp://ftp.ncbi.nih.gov/genbank/daily/Last.Releaseftp://ftp.ncbi.nih.gov/ncbi-asn1/daily/Last.Release
contain the number of the GenBank release which is currently installed
on the NCBI FTP site. As of Release 142.0 in June 2004, these files will
be moved and renamed as:
ftp://ftp.ncbi.nih.gov/genbank/GB_Release_Numberftp://ftp.ncbi.nih.gov/ncbi-asn1/GB_Release_Number
The /daily subdirectories, which had been used for cumulative update
products that are no longer supported, will be deleted at that time.
---
- gttaacaattaaagagtgtttatcgaaattcattatatagtggtttatatagaccacttc
-
- GenBank newsgroup see: http://www.bio.net/hypermail/genbankb/
- GENBANKB e-mail: messages sent to genbankb at net.bio.net
- subscribe: e-mail biosci-server at net.bio.net with: subscribe genbankb
- unsub: e-mail biosci-server at net.bio.net with: unsubscribe genbankb
- GenBank on the WWW, see: http://www.ncbi.nlm.nih.gov/Genbank/
- problems with GENBANKB? E-mail moderator: francis at bioinformatics.ubc.ca
