The complete SIFTER installation is a complex interaction of data and software. The first time it's easy for things to fail because you have something specified incorrectly or are missing some key component. Figuring out what is happening (or not happening most likely) may not be easy.
The demo system install process will provide a chance to check all of this out. As each piece is installed, test programs are run to verify each step, so you will be able to determine precisely what's not working.
When you begin this install, you must have installed Perl, the Perl modules and the database software. You should have also created a database for the demo system and have a database userid and password to manage the tables for the database. We only assume you have extracted the SIFTER files into some directory. You need not have installed SIFTER into production.
You should, however, have read the entire install documentation as you will need to choose directories for the realm files and you'll need to know where the web server's HTDOCS directory can be found. You'll also need to have write access to all the directories you provide.
The entire process is driven by a Perl script which prompts you for all the details described in the install document. Here's your chance to see how well you understand the instructions. :-)
As each step of the install is done, separate Perl scripts are run to verify the following:
Simple tests are made at each step to verify the system appears to be properly installed. We do not make extensive tests here, but we assume if the basics work, then the whole system will work. This is not a regression test, at least not yet.
The entire process is driven by the script perl/scripts/installdemo.pl. As you correct failures, the script should be restarted to continue. You'll want to invoke installdemo.pl with options to avoid repeating some steps (especially the step which asks for the realm information). To see the documentation on these options, invoke the command perl installdemo.pl -h.
To install the demo project, you must first initialize the SIFTER database and load data into it. This will take 5-10 minutes and will require you to provide the details about your database. After this is working, you will install the demo software in your web server's HTDOCS directory so you may use the web browser.
Let's get started! Lines you enter are shown in red and commentary is shown in blue. Also note that some of the following was reformatted to make it easier to read comments.
cd [whereever]/sifter-1.00
Start in SIFTER distribution directory
perl -w perl/scripts/installdemo.pl
If your database is Sybase, you'll want the environment
varibale SYBASE set. You may also need to specify a special
Perl library directory. If so, set these environment
variables or specify something like this:
(PERL5LIB=blahblah; export PERL5LIB; \
perl -w perl/scripts/installdemo.pl)
+----------------------------------------------------
| Install the Sifter Demo
+----------------------------------------------------
As many of the names as possible are going to be fixed
in this install, but there are some things we need from
you. Here's what we'll need:
(1) You need to have created a Sybase database where the Sifter
demo data will be stored. It need only be 2MB in size.
You need to know:
* the complete DSN for your database (an example is provided),
* the database userid
* the password for this database userid
(2) We need to know a web server where the SIFTER demo will
run. We will insist this be put in HTDOCS/sifter-demo.
(3) If the URL you provided in (2) requires a userid/password
to access the data, you will need to know
* web userid to use
* web password to use
(4) You will need to provide a directory where some of this
configuration information can be saved. We'll help by forcing
some things, but the harder part is that you must provide
a directory where a CGI running on the web server (from (2))
can read this information. Be careful here because this
information contains a database userid and password, so you
want it protected so that the web server can read it, but
not by anyone else except administrators.
Are you ready to try installing the Sifter demo?
Enter y or n:
y
Avoid this with the -quiet option the next time.
Do read it at least once so you know what is needed.
No point in going further if you don't know the
answers to what's going to be asked.
'use CGI' was found
'use Cwd' was found
etc.
You have installed all the Perl modules we need.
SOAP appeared to work
XML::Parser appeared to work
If anything fails here, you don't have the Perl
modules installed. If you do, but the modules are
installed in a non-standard place, set the
environment variable PERL5LIB to the directory
path for the modules and start again.
Provide the database DSN (e.g. 'dbi:Sybase:server=my_host;database=sifter_db'):
dbi:Sybase:server=SOMESERVER;database=SOMEDATABASE
Your DBA can tell you the Sybase 'server' and
'database' values.
Provide the database userid: DBUSERID
Provide the database password for this userid: DBPASSWORD
Again your DBA can tell you database user and
password that was defined.
Provide the host name of a web server where the demo will run: www.YOURDOMAIN.com
Your web master can tell you what this should
be if you don't know it.
Provide the protocol to access 'www.YOURDOMAIN.com' (https or enter for http): https
Again, ask your web master can tell you what
this should be if you don't know it.
Enter any special port in the URL to access 'www.YOURDOMAIN.com' (NNNN or enter):
Most times you should just press enter. Ask
your web master if you don't know.
If 'https://www.YOURDOMAIN.com/sifter-demo' is password protected,
provide the userid or press enter for none: MYUSER
If accessing the URL mentioned two lines above
will required a password, provide the userid
here. Again, ask your web master.
Provide the password or press enter for none: MYPASSWORD
Likewise.
Provide the full path to the HTDOCS for 'https://www.YOURDOMAIN.com/':HTDOCS
This is the directory where your web server
documents are kept. We need to create a directory
'sifter-demo' here and copy files so the demo works.
Ask your web master if you don't understand this.
We use this information in the second step of the
demo install.
We need a directory where the server information can be saved.
This directory must be readable by a CGI runnning on
https://www.YOURDOMAIN.com/sifter-demo
Enter directory path: /SOMEPATH/sifter/server
This is particularly important. This is a
directory where the database information
will be saved. This is a directory where
the file 'demo' will be created (since
the name of this project is 'demo').
Let's check this one last time:
dsn=dbi:Sybase:server=SOMESERVER;database=SOMEDATABASE
dbuid=DBUSERID dbpwd=DBPASSWORD
URL=https://www.YOURDOMAIN.com/sifter-demo
webuid=MYUSER webpwd=MYPASSWORD
CGI database details will be in '/SOMEPATH/sifter/server'
HTDOCS is in 'HTDOCS'
Shall we make the various files? y|n y
Check the values over carefully. If anything is
wrong, enter 'n' and start again. If any errors
are made here, it'll insure something will fail.
Creating client realm information...
Created client realm file '$HOME/.sifter/Sifter_Demo/demo'
We have arbitrarily chosen to save your client
realm files in a directory in your $HOME directory.
When all of this is over, you may delete this
directory. When you make a production system,
you'll want to choose a public directory that
everyone on your system can read.
Creating server realm information...
Created server realm file '/SOMEPATH/sifter/server/demo'
This says the server realm directory was
writable by you. We'll find out later if
the CGI scripts can read from it.
Creating user information to access the URL...
Created user realm file '$HOME/.sifter/user.cfg'
This is where we save the web userid and password
to access your URL.
Saved pointers to realm data in '/tmp/perlscriptsinstalldemo.pl.data'
Retrieving realm details from '/tmp/perlscriptsinstalldemo.pl.data'
Some of the input to installdemo.pl is saved here.
After you are done with the demo, this may be
deleted. If it gets removed, you'll have to enter
everything again.
Verifying DBI access to database works...
DSN=dbi:Sybase:server=SOMESERVER;database=SOMEDATABASE
UID=DBUSERID
PWD=DBPASSWORD
Connected to database: dbi:Sybase:server=SOMESERVER;database=SOMEDATABASE
CREATE table 'bogosity'
INSERT OK
UPDATE OK
SELECT OK
DELETE OK
DROP OK
DBI access to database appears to work
At this point we have verified that the Perl DBI
code works. Your database userid and password can
create/drop tables and put data in and get it out.
Destroy the SIFTER database and re-initialize it...
WARNING: destroy all data in Sifter realm 'demo' (dbtype=SYBASE)!
Proceed? (y/N) y
Now we are preparing to initialize the SIFTER data
in your database.
Running: perl/scripts/../../db/sybase/clean.sql
Running: perl/scripts/../../db/sybase/schema.sql
Running: perl/scripts/../../db/sybase/proc.sql
Running: perl/scripts/../../db/sybase/data.sql
Database for realm 'demo' has been initialized.
If successful, the SQL has been run using the
database userid and password and we've created
a bunch of tables.
Adding default attributes:
Added analysis attributes
Added map attributes
Added enumerations
SIFTER database successfully re-initialized. See log file '/tmp/installdemo.pl.sql.log'
Now we've loaded some SIFTER internal data into
the database. We're about ready to load some
real data in.
Verifying SIFTER SERVER database access works...
SIFTER SERVER database access appears to work
SIFTER SERVER access is fine. See log file '/tmp/installdemo.pl.checkdb.log'
We have used the SIFTER realm information to
access the initialized tables.
Loading demo Maps into database... Will take 1-3 minutes
Maps loaded into database. See log file '/tmp/installdemo.pl.adddemomaps.log'
We have used SIFTER code to load the samples/Maps
into the database.
Loading demo Results into database... This can take 5-15 minutes
Results loaded into database. See log file '/tmp/installdemo.pl.adddemoresults.log'
We're going to load over sixty results into the
database. As with the Maps, we will try to load
the samples/Results into the database.
If you've successfully gotten this far, congratulations are in order. This says many things are working: Perl, Perl modules, the database and much of the SIFTER code. The major thing that has not been tested yet is the SIFTER code running as CGI scripts on your web server. In theory this should not be difficult to make work since you got it all working for your own userid. But web servers can be tricky, so now we install and test the web interface.
cd [whereever]/sifter-1.00
Start in SIFTER distribution directory
perl -w perl/scripts/installdemo.pl -web
Notice the option '-web'. It makse all the
difference. If you set PERL5LIB earlier, you must
do so again. You do not need database environment
variables set.
Retrieving realm details from '/tmp/installdemo.data'
Cleaning SIFTER for install. Ignore errors.
Creating Makefiles starting at '/home/YOURUSER/dev/sifter'
Makefiles were created. See log file '/tmp/installdemo.serverinstall.log'
We start by finding where you have installed the
SIFTER code and then do 'perl Makefile.PL'.
Since you have provided all the details earlier,
you do not need to answer the questions.
Preparing SIFTER for install...
Note: Alert.java uses or overrides a deprecated API.
Note: Recompile with -deprecation for details.
Ignore these java compiler warnings.
Note: ./ClosableFrame.java uses or overrides a deprecated API.
Note: Recompile with -deprecation for details.
etc.
SIFTER ready for install. See log file '/tmp/installdemo.serverinstall.log'
'make server' has compiled everything and created
modules with the paths you provided.
Installing SIFTER...
Install was successful. See log file '/tmp/installdemo.serverinstall.log'
Everything is copied to the HTDOCS you specified.
You should have a running SIFTER installation,
but we'll check anyway :-)
Check the web server...
Web server is responding.
All CGI activity is logged to '/tmp/installdemo.cgi.log'
This just means we were able to retrieve a web
page from the demo web site. This means your web
url, userid and password are correct.
Check CGI scripts...
CGI scripts can be invoked.
CGI scripts can be invoked on this web server.
This means the server is configured to invoke
files with a 'cgi' extension as CGI scripts.
Check that CGI scripts can access all modules...
CGI scripts can access all modules.
Your CGI scripts can access all the Perl modules
SIFTER will require.
Check CGI scripts using CGI module...
CGI scripts can use the CGI module.
Your CGI scripts can actually use the CGI module.
Check CGI scripts using DBI module...
CGI scripts can use the DBI module.
Your CGI scripts can use the DBI module to access
the database. The same database tests we did earlier
are run.
Check CGI scripts using SIFTER module...
CGI scripts can use the SIFTER module.
We could access some SIFTER information on the
web server.
Your SIFTER demo site appears to work
URL=https://www.YOURDOMAIN.com/sifter-demo
All tests have completed for the SIFTER demo site
We can't think of anything more to test. In your
web browser, visit the url and select the java
application (select the 'Start' button). Java
is the only thing left to try out.
The SIFTER demo should work for your users. Use your web browser to access the url (e.g. https://www.YOURDOMAIN.com/sifter-demo). Select the 'Start' button and you should see a Java applet load and open a window. Try a query for results on chromosomes 6, 11, 20 or 22. Select some results and plot them and get familiar with the interface.
You should also select the other links at the top of the first page (Analysis Attr, Q_Maps and Map Attr) and see that they behave. Look at the maps and attributes and see how SIFTER defines and uses attributes.
When you are satisified you understand what's going on,
it's time to create your own project.
When the Demo Goes Wrong
SIFTER requires a number of components to work smoothly together. If this is your first installation (or first installation on this machine), you may well have a few surprises. To help assess problem areas, we provde the demo installation. You are encouraged to install the demo. It won't take long and it has the great advantage that it will do a sanity check of each of the software components. The following is a description of the checks made by the demo and if it fails, what you should investigate.
SIFTER requires a working installation of Perl. While we require version 5.6, you may well find that earlier version of 5.x will work for you. This requirement should not be a burden for modern Unix systems, as almost every system we've seen has Perl. Hope this includes you :-)
The command 'perl' should be in your PATH - again this should not be a surprise. You should also have the basic Perl modules installed (especially File::* and Getopt::Long). Again, this should be in every standard install.
The first command is 'perl installdemo.pl'. If you have Perl installed as explained above, this should compile. If this works, we have an excellent chance to test your environment.
The first check of your system is to verify all of the Perl modules needed for SIFTER are installed. This assumes they are installed in your default Perl libraries or you have PERL5LIB set. If the latter is the case, you must be sure that PERL5LIB is set for cases where SIFTER code is run (either clients loading results into the database or for the CGI scripts running on the web server).
In order to access your database directly, some database environments will require you to set an environment variable. For instance, Sybase requires that SYBASE be set. This should be no surprise for your installation.
The next check is to see if the Perl DBI code can connect to your database. This could fail because you don't have DBI installed completely or because the DSN, database userid or password is not set properly. In this same test, the Perl code will create a table, insert some data, update it, delete a row and drop the table -- all things the database userid you provide must be able to do.
Next we create and initialize all the internal SIFTER tables. Depending on your database, this requires a program that comes with your database to read the SQL and do the proper work. This might require environment variables to be set or the path to the program to be included.
Now we load a small number of maps and results into the demo database. This could fail because log files fill up, or the space for the data is inadequate.
By this point we've tested the SIFTER client code to verify the basic components are there and work. This has not been much more than a cursory sanity check. There are still plenty of environmental problems that could show up later (think of database space problems), but it's a good start.
The next phase of installdemo.pl offers to check your SIFTER web site. This can fail in several non-intuitive ways. Your web master should have done some things. We begin by checking the basics.
The first step is to 'make' the server installation. Compiling the Java code is probably the most likely thing to fail in the make. It could also fail to install because you do not have write access to the sifter-demo directory in HTDOCS (where all the web source files are saved).
After the install, we start by trying to fetch a web page. This can fail because the URL is incorrect, the protocol (http or https) was wrong, the server name was wrong or you specified HTDOCS incorrectly or the directory sifter-demo is not accessible to the web server. This can also fail because you specified a web userid and password (saved in $HOME/.sifter/user.cfg) and these values were incorrect.
The next check is that a CGI script can be invoked. This verifies the web master set up CGI scripts as was expected.. This also means that Perl is available to the web server. If this does not work, look in the error log for your web server (ask your web master).
We then check that the CGI module works for CGI scripts. If this is the case, we expect all the normal Perl modules to work. CGI failures are logged to the web servers log files. You may need to ask your web master to help figure out what's not going on if this fails.
Next we try to access the database directly. This verifies that DBI works and that the DSN and database userid and password work.
Our last CGI test is to see if the SIFTER code can access the database using the realm information you provided. This could fail because the web server does not have read access to the server directory you provided during the install. The path to this directory can be found in sifter-demo/modules/Sifter/Config/Server.pm in your HTDOCS directory. Once this works, we can be pretty confident the web server is ready.
The last test is for you to use your web browser to access the web pages. The 'Start' button should launch a Java applet. This could fail because your browser does not have Java enabled. You may want to look at the java console for information on errors (in Netscape look on the Communicator -> Tools menu).
Version=$Id: demo.html,v 1.9 2002/11/22 16:41:26 tpg Exp $