Method to do indexing @production server

Reinhard Doelz doelz at comp.bioz.unibas.ch
Wed May 24 12:04:05 EST 1995

we need to index data while the server is expected to answer replies. As 
the normal procedure of srscheck/srsupdate will invalidate indices during
the process of srsbuild, we came up with the following solution. Maybe you 
have found a better way of doing it, we'd like to hear from you! 

The following still assumes that the data are not needed, only the indices. 
This is OK as long as searches or SRS operations need only the indices.
This assumption is only true as a rough approximation. Procedures could be 
developed which handle this, too. 


1. Buy disks. 
2. Make a scratch dir and symlink each entry in SRSINX. 
3. Parse output from srscheck and delete all symlinks to files which are  
   to be rebuilt. 
4. reassign SRSINX and rebuild indices. 
5. Delete all symlinks in the scratch directory. 
6. copy all newly created indices into production SRSINX. 

The following shell does this, exemplarically. 
In order to map logical names to file names, the following awk script is 
needed (must be modified, too).

============================> map.awk <=====================================
  if ($1 == "XEMBL")     {  print "xembl"      } 
  if ($1 == "XXEMBL")    {  print "xxembl"     }
  if ($1 == "EMNEW")     {  print "emnew"      }
  if ($1 == "GBNEW")     {  print "gbnew"      }
  if ($1 == "GBNEW_EXCL"){  print "gbnew_excl" }
  if ($1 == "TREMBLNEW") {  print "tremblnew"  }
=============================> end of map.awk <==============================

=========================> update.csh <===================================
unlimit                                              # common SRS practice
setenv SCRATCHBIN /bioz1/software/www/SRSUPDATE      # location of map.awk
setenv SCRATCHINX /biox4/srs                         # scratch directory

source /bioz1/software/www/SRS/etc/prep_srs          # srs prep

#----------------------------------- create linkages 
        cd $SRSINX
        foreach i (*)
                ln -s $SRSINX/$i $SCRATCHINX/$i
        cd $SCRATCHINX
#------------------------------------ see, is there something to be done? 
        srscheck >& log 
        cat log | grep must | nawk '{n = split ($0,field,"\""); print field[2]}' | nawk -f $SCRATCHBIN/map.awk | sort -u  > log2
#----------------------------------- delete all links which have these 
        unalias ls
        foreach i ( `cat log2`) 
                rm $SCRATCHINX/$i''*
#---------------------------------- establish working index 
        setenv SRSSAV $SRSINX
        setenv SRSINX $SCRATCHINX
#---------------------------------- redo the srscheck and do srsupdate
#                                   (dont reassign SRSINX in prep_srs)
        cat `which srsupdate`  | grep -v prep_srs > $SCRATCHBIN/pilot.csh
        source $SCRATCHBIN/pilot.csh
#---------------------------------- remove all files which are symlinks 
        cd $SCRATCHINX
        find . -type l -exec rm '{}' 
#---------------------------------- copy newly survived index files 
        foreach i (*)
                cp $i $SRSSAV

=========================> end of update.csh <===============================

I forgot to mention that this works only on UNIX. 


 R.Doelz         Klingelbergstr.70| Tel. x41 61 267 2247  Fax x41 61 267 2078|
 Biocomputing        CH 4056 Basel| electronic Mail    doelz at ubaclu.unibas.ch|
 Biozentrum der Universitaet Basel|-------------- Switzerland ---------------|
<a href=http://beta.embnet.unibas.ch/>EMBnet Switzerland:info at ch.embnet.org</a> 

More information about the Bio-srs mailing list

Send comments to us at biosci-help [At] net.bio.net