Method to do indexing @production server

[Reinhard Doelz]

Colleagues, 
we need to index data while the server is expected to answer replies. As 
the normal procedure of srscheck/srsupdate will invalidate indices during
the process of srsbuild, we came up with the following solution. Maybe you 
have found a better way of doing it, we'd like to hear from you! 

Note: 
The following still assumes that the data are not needed, only the indices. 
This is OK as long as searches or SRS operations need only the indices.
This assumption is only true as a rough approximation. Procedures could be 
developed which handle this, too. 

Principle: 

1. Buy disks. 
2. Make a scratch dir and symlink each entry in SRSINX. 
3. Parse output from srscheck and delete all symlinks to files which are  
   to be rebuilt. 
4. reassign SRSINX and rebuild indices. 
5. Delete all symlinks in the scratch directory. 
6. copy all newly created indices into production SRSINX. 

The following shell does this, exemplarically. 
In order to map logical names to file names, the following awk script is 
needed (must be modified, too).

============================> map.awk <=====================================
{
  if ($1 == "XEMBL")     {  print "xembl"      } 
  if ($1 == "XXEMBL")    {  print "xxembl"     }
  if ($1 == "EMNEW")     {  print "emnew"      }
  if ($1 == "GBNEW")     {  print "gbnew"      }
  if ($1 == "GBNEW_EXCL"){  print "gbnew_excl" }
  if ($1 == "TREMBLNEW") {  print "tremblnew"  }
} 
=============================> end of map.awk <==============================

=========================> update.csh <===================================
#!/bin/csh 
unlimit                                              # common SRS practice
setenv SCRATCHBIN /bioz1/software/www/SRSUPDATE      # location of map.awk
setenv SCRATCHINX /biox4/srs                         # scratch directory

source /bioz1/software/www/SRS/etc/prep_srs          # srs prep

#
#----------------------------------- create linkages 
#
        cd $SRSINX
        foreach i (*)
                ln -s $SRSINX/$i $SCRATCHINX/$i
                end
        cd $SCRATCHINX
#
#------------------------------------ see, is there something to be done? 
        srscheck >& log 
        cat log | grep must | nawk '{n = split ($0,field,"\""); print field[2]}' | nawk -f $SCRATCHBIN/map.awk | sort -u  > log2
#
#----------------------------------- delete all links which have these 
#
        unalias ls
        foreach i ( `cat log2`) 
                rm $SCRATCHINX/$i''*
                end
#
#---------------------------------- establish working index 
#
        setenv SRSSAV $SRSINX
        setenv SRSINX $SCRATCHINX
#
#---------------------------------- redo the srscheck and do srsupdate
#                                   (dont reassign SRSINX in prep_srs)
        srscheck 
        cat `which srsupdate`  | grep -v prep_srs > $SCRATCHBIN/pilot.csh
        source $SCRATCHBIN/pilot.csh
#
#---------------------------------- remove all files which are symlinks 
#
        cd $SCRATCHINX
        find . -type l -exec rm '{}' 
#
#---------------------------------- copy newly survived index files 
#
        foreach i (*)
                cp $i $SRSSAV
                end


=========================> end of update.csh <===============================

I forgot to mention that this works only on UNIX. 

Regards
Reinhard


-- 
 R.Doelz         Klingelbergstr.70| Tel. x41 61 267 2247  Fax x41 61 267 2078|
 Biocomputing        CH 4056 Basel| electronic Mail    doelz at ubaclu.unibas.ch|
 Biozentrum der Universitaet Basel|-------------- Switzerland ---------------|
<a href=http://beta.embnet.unibas.ch/>EMBnet Switzerland:info at ch.embnet.org</a>