SIFTER Demo Installation

The complete SIFTER installation is a complex interaction of data and software. The first time it's easy for things to fail because you have something specified incorrectly or are missing some key component. Figuring out what is happening (or not happening most likely) may not be easy.

The demo system install process will provide a chance to check all of this out. As each piece is installed, test programs are run to verify each step, so you will be able to determine precisely what's not working.

Requirements

When you begin this install, you must have installed Perl, the Perl modules and the database software. You should have also created a database for the demo system and have a database userid and password to manage the tables for the database. We only assume you have extracted the SIFTER files into some directory. You need not have installed SIFTER into production.

You should, however, have read the entire install documentation as you will need to choose directories for the realm files and you'll need to know where the web server's HTDOCS directory can be found. You'll also need to have write access to all the directories you provide.

Process

The entire process is driven by a Perl script which prompts you for all the details described in the install document. Here's your chance to see how well you understand the instructions. :-)

As each step of the install is done, separate Perl scripts are run to verify the following:

Simple tests are made at each step to verify the system appears to be properly installed. We do not make extensive tests here, but we assume if the basics work, then the whole system will work. This is not a regression test, at least not yet.

Installing the Project 'demo'

The entire process is driven by the script perl/scripts/installdemo.pl. As you correct failures, the script should be restarted to continue. You'll want to invoke installdemo.pl with options to avoid repeating some steps (especially the step which asks for the realm information). To see the documentation on these options, invoke the command perl installdemo.pl -h.

To install the demo project, you must first initialize the SIFTER database and load data into it. This will take 5-10 minutes and will require you to provide the details about your database. After this is working, you will install the demo software in your web server's HTDOCS directory so you may use the web browser.

Let's get started! Lines you enter are shown in red and commentary is shown in blue. Also note that some of the following was reformatted to make it easier to read comments.

Installing demo - Step One

cd [whereever]/sifter-1.00

                    Start in SIFTER distribution directory

perl -w perl/scripts/installdemo.pl 

                    If your database is Sybase, you'll want the environment
                    varibale SYBASE set. You may also need to specify a special
                    Perl library directory. If so, set these environment
                    variables or specify something like this:
                      (PERL5LIB=blahblah; export PERL5LIB; \
                         perl -w perl/scripts/installdemo.pl)


+----------------------------------------------------
|   Install the Sifter Demo
+----------------------------------------------------
As many of the names as possible are going to be fixed 
in this install, but there are some things we need from
you. Here's what we'll need:

(1) You need to have created a Sybase database where the Sifter
demo data will be stored. It need only be 2MB in size.
You need to know:
    * the complete DSN for your database (an example is provided), 
    * the database userid
    * the password for this database userid

(2) We need to know a web server where the SIFTER demo will 
run. We will insist this be put in HTDOCS/sifter-demo.

(3) If the URL you provided in (2) requires a userid/password
to access the data, you will need to know
    * web userid to use
    * web password to use

(4) You will need to provide a directory where some of this
configuration information can be saved. We'll help by forcing
some things, but the harder part is that you must provide
a directory where a CGI running on the web server (from (2))
can read this information. Be careful here because this
information contains a database userid and password, so you
want it protected so that the web server can read it, but
not by anyone else except administrators.

Are you ready to try installing the Sifter demo?
Enter y or n:
y 
                    Avoid this with the -quiet option the next time.
                    Do read it at least once so you know what is needed.
                    No point in going further if you don't know the
                    answers to what's going to be asked.

'use CGI' was found
'use Cwd' was found
etc.

You have installed all the Perl modules we need.
SOAP appeared to work
XML::Parser appeared to work

                    If anything fails here, you don't have the Perl
                    modules installed. If you do, but the modules are
                    installed in a non-standard place, set the
                    environment variable PERL5LIB to the directory
                    path for the modules and start again.

Provide the database DSN (e.g. 'dbi:Sybase:server=my_host;database=sifter_db'):
dbi:Sybase:server=SOMESERVER;database=SOMEDATABASE

                    Your DBA can tell you the Sybase 'server' and
                    'database' values.
                    
Provide the database userid: DBUSERID
Provide the database password for this userid: DBPASSWORD

                    Again your DBA can tell you database user and
                    password that was defined.

Provide the host name of a web server where the demo will run: www.YOURDOMAIN.com

                    Your web master can tell you what this should
                    be if you don't know it.

Provide the protocol to access 'www.YOURDOMAIN.com' (https or enter for http): https

                    Again, ask your web master can tell you what
                    this should be if you don't know it.

Enter any special port in the URL to access 'www.YOURDOMAIN.com' (NNNN or enter): 

                    Most times you should just press enter. Ask
                    your web master if you don't know.

If 'https://www.YOURDOMAIN.com/sifter-demo' is password protected,
    provide the userid or press enter for none: MYUSER

                    If accessing the URL mentioned two lines above
                    will required a password, provide the userid
                    here. Again, ask your web master.

Provide the password or press enter for none: MYPASSWORD

                    Likewise.

Provide the full path to the HTDOCS for 'https://www.YOURDOMAIN.com/':HTDOCS

                    This is the directory where your web server 
                    documents are kept. We need to create a directory
                    'sifter-demo' here and copy files so the demo works.
                    Ask your web master if you don't understand this.
                    We use this information in the second step of the
                    demo install.

We need a directory where the server information can be saved.
This directory must be readable by a CGI runnning on
    https://www.YOURDOMAIN.com/sifter-demo
Enter directory path: /SOMEPATH/sifter/server

                    This is particularly important. This is a
                    directory where the database information
                    will be saved. This is a directory where
                    the file 'demo' will be created (since
                    the name of this project is 'demo').

Let's check this one last time:
  dsn=dbi:Sybase:server=SOMESERVER;database=SOMEDATABASE
    dbuid=DBUSERID dbpwd=DBPASSWORD
  URL=https://www.YOURDOMAIN.com/sifter-demo
    webuid=MYUSER  webpwd=MYPASSWORD
  CGI database details will be in '/SOMEPATH/sifter/server'
  HTDOCS is in 'HTDOCS'
 
Shall we make the various files? y|n y

                    Check the values over carefully. If anything is
                    wrong, enter 'n' and start again. If any errors 
                    are made here, it'll insure something will fail.

Creating client realm information...
  Created client realm file '$HOME/.sifter/Sifter_Demo/demo'

                    We have arbitrarily chosen to save your client
                    realm files in a directory in your $HOME directory.
                    When all of this is over, you may delete this
                    directory. When you make a production system,
                    you'll want to choose a public directory that
                    everyone on your system can read.

Creating server realm information...
  Created server realm file '/SOMEPATH/sifter/server/demo'

                    This says the server realm directory was
                    writable by you. We'll find out later if
                    the CGI scripts can read from it.

Creating user information to access the URL...
  Created user realm file '$HOME/.sifter/user.cfg'

                    This is where we save the web userid and password
                    to access your URL.

Saved pointers to realm data in '/tmp/perlscriptsinstalldemo.pl.data'
Retrieving realm details from '/tmp/perlscriptsinstalldemo.pl.data'

                    Some of the input to installdemo.pl is saved here.
                    After you are done with the demo, this may be
                    deleted. If it gets removed, you'll have to enter
                    everything again.

Verifying DBI access to database works...
  DSN=dbi:Sybase:server=SOMESERVER;database=SOMEDATABASE
  UID=DBUSERID
  PWD=DBPASSWORD
  Connected to database: dbi:Sybase:server=SOMESERVER;database=SOMEDATABASE
  CREATE table 'bogosity'
  INSERT OK
  UPDATE OK
  SELECT OK
  DELETE OK
  DROP OK
DBI access to database appears to work

                    At this point we have verified that the Perl DBI
                    code works. Your database userid and password can
                    create/drop tables and put data in and get it out.

Destroy the SIFTER database and re-initialize it...
WARNING: destroy all data in Sifter realm 'demo' (dbtype=SYBASE)!
Proceed? (y/N) y

                    Now we are preparing to initialize the SIFTER data
                    in your database.

  Running: perl/scripts/../../db/sybase/clean.sql
  Running: perl/scripts/../../db/sybase/schema.sql
  Running: perl/scripts/../../db/sybase/proc.sql
  Running: perl/scripts/../../db/sybase/data.sql

Database for realm 'demo' has been initialized.

                    If successful, the SQL has been run using the
                    database userid and password and we've created
                    a bunch of tables.

Adding default attributes:
Added analysis attributes
Added map attributes
Added enumerations
SIFTER database successfully re-initialized. See log file '/tmp/installdemo.pl.sql.log'

                    Now we've loaded some SIFTER internal data into
                    the database. We're about ready to load some
                    real data in.

Verifying SIFTER SERVER database access works...
SIFTER SERVER database access appears to work
SIFTER SERVER access is fine. See log file '/tmp/installdemo.pl.checkdb.log'

                    We have used the SIFTER realm information to
                    access the initialized tables.

Loading demo Maps into database... Will take 1-3 minutes
Maps loaded into database. See log file '/tmp/installdemo.pl.adddemomaps.log'

                    We have used SIFTER code to load the samples/Maps
                    into the database.

Loading demo Results into database... This can take 5-15 minutes
Results loaded into database. See log file '/tmp/installdemo.pl.adddemoresults.log'

                    We're going to load over sixty results into the
                    database. As with the Maps, we will try to load
                    the samples/Results into the database.

Installing demo - Step Two

If you've successfully gotten this far, congratulations are in order. This says many things are working: Perl, Perl modules, the database and much of the SIFTER code. The major thing that has not been tested yet is the SIFTER code running as CGI scripts on your web server. In theory this should not be difficult to make work since you got it all working for your own userid. But web servers can be tricky, so now we install and test the web interface.

cd [whereever]/sifter-1.00

                    Start in SIFTER distribution directory

perl -w perl/scripts/installdemo.pl -web 

                    Notice the option '-web'. It makse all the 
                    difference. If you set PERL5LIB earlier, you must
                    do so again. You do not need database environment
                    variables set.

Retrieving realm details from '/tmp/installdemo.data'
Cleaning SIFTER for install. Ignore errors.
Creating Makefiles starting at '/home/YOURUSER/dev/sifter'
Makefiles were created. See log file '/tmp/installdemo.serverinstall.log'

                    We start by finding where you have installed the
                    SIFTER code and then do 'perl Makefile.PL'.
                    Since you have provided all the details earlier,
                    you do not need to answer the questions.

Preparing SIFTER for install...
Note: Alert.java uses or overrides a deprecated API.
Note: Recompile with -deprecation for details.
                    Ignore these java compiler warnings.
Note: ./ClosableFrame.java uses or overrides a deprecated API.
Note: Recompile with -deprecation for details.
etc.
SIFTER ready for install. See log file '/tmp/installdemo.serverinstall.log'

                    'make server' has compiled everything and created
                    modules with the paths you provided.

Installing SIFTER...
Install was successful. See log file '/tmp/installdemo.serverinstall.log'

                    Everything is copied to the HTDOCS you specified.
                    You should have a running SIFTER installation,
                    but we'll check anyway :-)

Check the web server...
Web server is responding.
All CGI activity is logged to '/tmp/installdemo.cgi.log'

                    This just means we were able to retrieve a web
                    page from the demo web site. This means your web
                    url, userid and password are correct.

Check CGI scripts...
CGI scripts can be invoked.

                    CGI scripts can be invoked on this web server.
                    This means the server is configured to invoke
                    files with a 'cgi' extension as CGI scripts.

Check that CGI scripts can access all modules...
CGI scripts can access all modules.

                    Your CGI scripts can access all the Perl modules
                    SIFTER will require.

Check CGI scripts using CGI module...
CGI scripts can use the CGI module.

                    Your CGI scripts can actually use the CGI module.

Check CGI scripts using DBI module...
CGI scripts can use the DBI module.

                    Your CGI scripts can use the DBI module to access
                    the database. The same database tests we did earlier
                    are run.

Check CGI scripts using SIFTER module...
CGI scripts can use the SIFTER module.

                    We could access some SIFTER information on the
                    web server.

Your SIFTER demo site appears to work
  URL=https://www.YOURDOMAIN.com/sifter-demo

All tests have completed for the SIFTER demo site

                    We can't think of anything more to test. In your 
                    web browser, visit the url and select the java
                    application (select the 'Start' button). Java
                    is the only thing left to try out.

The SIFTER demo should work for your users. Use your web browser to access the url (e.g. https://www.YOURDOMAIN.com/sifter-demo). Select the 'Start' button and you should see a Java applet load and open a window. Try a query for results on chromosomes 6, 11, 20 or 22. Select some results and plot them and get familiar with the interface.

You should also select the other links at the top of the first page (Analysis Attr, Q_Maps and Map Attr) and see that they behave. Look at the maps and attributes and see how SIFTER defines and uses attributes.

When you are satisified you understand what's going on, it's time to create your own project.

When the Demo Goes Wrong

SIFTER requires a number of components to work smoothly together. If this is your first installation (or first installation on this machine), you may well have a few surprises. To help assess problem areas, we provde the demo installation. You are encouraged to install the demo. It won't take long and it has the great advantage that it will do a sanity check of each of the software components. The following is a description of the checks made by the demo and if it fails, what you should investigate.

SIFTER requires a working installation of Perl. While we require version 5.6, you may well find that earlier version of 5.x will work for you. This requirement should not be a burden for modern Unix systems, as almost every system we've seen has Perl. Hope this includes you :-)

The command 'perl' should be in your PATH - again this should not be a surprise. You should also have the basic Perl modules installed (especially File::* and Getopt::Long). Again, this should be in every standard install.

The first command is 'perl installdemo.pl'. If you have Perl installed as explained above, this should compile. If this works, we have an excellent chance to test your environment.

The first check of your system is to verify all of the Perl modules needed for SIFTER are installed. This assumes they are installed in your default Perl libraries or you have PERL5LIB set. If the latter is the case, you must be sure that PERL5LIB is set for cases where SIFTER code is run (either clients loading results into the database or for the CGI scripts running on the web server).

In order to access your database directly, some database environments will require you to set an environment variable. For instance, Sybase requires that SYBASE be set. This should be no surprise for your installation.

The next check is to see if the Perl DBI code can connect to your database. This could fail because you don't have DBI installed completely or because the DSN, database userid or password is not set properly. In this same test, the Perl code will create a table, insert some data, update it, delete a row and drop the table -- all things the database userid you provide must be able to do.

Next we create and initialize all the internal SIFTER tables. Depending on your database, this requires a program that comes with your database to read the SQL and do the proper work. This might require environment variables to be set or the path to the program to be included.

Now we load a small number of maps and results into the demo database. This could fail because log files fill up, or the space for the data is inadequate.

By this point we've tested the SIFTER client code to verify the basic components are there and work. This has not been much more than a cursory sanity check. There are still plenty of environmental problems that could show up later (think of database space problems), but it's a good start.

The next phase of installdemo.pl offers to check your SIFTER web site. This can fail in several non-intuitive ways. Your web master should have done some things. We begin by checking the basics.

The first step is to 'make' the server installation. Compiling the Java code is probably the most likely thing to fail in the make. It could also fail to install because you do not have write access to the sifter-demo directory in HTDOCS (where all the web source files are saved).

After the install, we start by trying to fetch a web page. This can fail because the URL is incorrect, the protocol (http or https) was wrong, the server name was wrong or you specified HTDOCS incorrectly or the directory sifter-demo is not accessible to the web server. This can also fail because you specified a web userid and password (saved in $HOME/.sifter/user.cfg) and these values were incorrect.

The next check is that a CGI script can be invoked. This verifies the web master set up CGI scripts as was expected.. This also means that Perl is available to the web server. If this does not work, look in the error log for your web server (ask your web master).

We then check that the CGI module works for CGI scripts. If this is the case, we expect all the normal Perl modules to work. CGI failures are logged to the web servers log files. You may need to ask your web master to help figure out what's not going on if this fails.

Next we try to access the database directly. This verifies that DBI works and that the DSN and database userid and password work.

Our last CGI test is to see if the SIFTER code can access the database using the realm information you provided. This could fail because the web server does not have read access to the server directory you provided during the install. The path to this directory can be found in sifter-demo/modules/Sifter/Config/Server.pm in your HTDOCS directory. Once this works, we can be pretty confident the web server is ready.

The last test is for you to use your web browser to access the web pages. The 'Start' button should launch a Java applet. This could fail because your browser does not have Java enabled. You may want to look at the java console for information on errors (in Netscape look on the Communicator -> Tools menu).

Version=$Id: demo.html,v 1.9 2002/11/22 16:41:26 tpg Exp $