The complete SIFTER installation is a complex interaction of data and software. The first time it's easy for things to fail because you have something specified incorrectly or are missing some key component. Figuring out what is happening (or not happening most likely) may not be easy.
The demo system install process will provide a chance to check all of this out. As each piece is installed, test programs are run to verify each step, so you will be able to determine precisely what's not working.
When you begin this install, you must have installed Perl, the Perl modules and the database software. You should have also created a database for the demo system and have a database userid and password to manage the tables for the database. We only assume you have extracted the SIFTER files into some directory. You need not have installed SIFTER into production.
You should, however, have read the entire install documentation as you will need to choose directories for the realm files and you'll need to know where the web server's HTDOCS directory can be found. You'll also need to have write access to all the directories you provide.
The entire process is driven by a Perl script which prompts you for all the details described in the install document. Here's your chance to see how well you understand the instructions. :-)
As each step of the install is done, separate Perl scripts are run to verify the following:
Simple tests are made at each step to verify the system appears to be properly installed. We do not make extensive tests here, but we assume if the basics work, then the whole system will work. This is not a regression test, at least not yet.
The entire process is driven by the script perl/scripts/installdemo.pl. As you correct failures, the script should be restarted to continue. You'll want to invoke installdemo.pl with options to avoid repeating some steps (especially the step which asks for the realm information). To see the documentation on these options, invoke the command perl installdemo.pl -h.
To install the demo project, you must first initialize the SIFTER database and load data into it. This will take 5-10 minutes and will require you to provide the details about your database. After this is working, you will install the demo software in your web server's HTDOCS directory so you may use the web browser.
Let's get started! Lines you enter are shown in red and commentary is shown in blue. Also note that some of the following was reformatted to make it easier to read comments.
cd [whereever]/sifter-1.00 Start in SIFTER distribution directory perl -w perl/scripts/installdemo.pl If your database is Sybase, you'll want the environment varibale SYBASE set. You may also need to specify a special Perl library directory. If so, set these environment variables or specify something like this: (PERL5LIB=blahblah; export PERL5LIB; \ perl -w perl/scripts/installdemo.pl) +---------------------------------------------------- | Install the Sifter Demo +---------------------------------------------------- As many of the names as possible are going to be fixed in this install, but there are some things we need from you. Here's what we'll need: (1) You need to have created a Sybase database where the Sifter demo data will be stored. It need only be 2MB in size. You need to know: * the complete DSN for your database (an example is provided), * the database userid * the password for this database userid (2) We need to know a web server where the SIFTER demo will run. We will insist this be put in HTDOCS/sifter-demo. (3) If the URL you provided in (2) requires a userid/password to access the data, you will need to know * web userid to use * web password to use (4) You will need to provide a directory where some of this configuration information can be saved. We'll help by forcing some things, but the harder part is that you must provide a directory where a CGI running on the web server (from (2)) can read this information. Be careful here because this information contains a database userid and password, so you want it protected so that the web server can read it, but not by anyone else except administrators. Are you ready to try installing the Sifter demo? Enter y or n: y Avoid this with the -quiet option the next time. Do read it at least once so you know what is needed. No point in going further if you don't know the answers to what's going to be asked. 'use CGI' was found 'use Cwd' was found etc. You have installed all the Perl modules we need. SOAP appeared to work XML::Parser appeared to work If anything fails here, you don't have the Perl modules installed. If you do, but the modules are installed in a non-standard place, set the environment variable PERL5LIB to the directory path for the modules and start again. Provide the database DSN (e.g. 'dbi:Sybase:server=my_host;database=sifter_db'): dbi:Sybase:server=SOMESERVER;database=SOMEDATABASE Your DBA can tell you the Sybase 'server' and 'database' values. Provide the database userid: DBUSERID Provide the database password for this userid: DBPASSWORD Again your DBA can tell you database user and password that was defined. Provide the host name of a web server where the demo will run: www.YOURDOMAIN.com Your web master can tell you what this should be if you don't know it. Provide the protocol to access 'www.YOURDOMAIN.com' (https or enter for http): https Again, ask your web master can tell you what this should be if you don't know it. Enter any special port in the URL to access 'www.YOURDOMAIN.com' (NNNN or enter): Most times you should just press enter. Ask your web master if you don't know. If 'https://www.YOURDOMAIN.com/sifter-demo' is password protected, provide the userid or press enter for none: MYUSER If accessing the URL mentioned two lines above will required a password, provide the userid here. Again, ask your web master. Provide the password or press enter for none: MYPASSWORD Likewise. Provide the full path to the HTDOCS for 'https://www.YOURDOMAIN.com/':HTDOCS This is the directory where your web server documents are kept. We need to create a directory 'sifter-demo' here and copy files so the demo works. Ask your web master if you don't understand this. We use this information in the second step of the demo install. We need a directory where the server information can be saved. This directory must be readable by a CGI runnning on https://www.YOURDOMAIN.com/sifter-demo Enter directory path: /SOMEPATH/sifter/server This is particularly important. This is a directory where the database information will be saved. This is a directory where the file 'demo' will be created (since the name of this project is 'demo'). Let's check this one last time: dsn=dbi:Sybase:server=SOMESERVER;database=SOMEDATABASE dbuid=DBUSERID dbpwd=DBPASSWORD URL=https://www.YOURDOMAIN.com/sifter-demo webuid=MYUSER webpwd=MYPASSWORD CGI database details will be in '/SOMEPATH/sifter/server' HTDOCS is in 'HTDOCS' Shall we make the various files? y|n y Check the values over carefully. If anything is wrong, enter 'n' and start again. If any errors are made here, it'll insure something will fail. Creating client realm information... Created client realm file '$HOME/.sifter/Sifter_Demo/demo' We have arbitrarily chosen to save your client realm files in a directory in your $HOME directory. When all of this is over, you may delete this directory. When you make a production system, you'll want to choose a public directory that everyone on your system can read. Creating server realm information... Created server realm file '/SOMEPATH/sifter/server/demo' This says the server realm directory was writable by you. We'll find out later if the CGI scripts can read from it. Creating user information to access the URL... Created user realm file '$HOME/.sifter/user.cfg' This is where we save the web userid and password to access your URL. Saved pointers to realm data in '/tmp/perlscriptsinstalldemo.pl.data' Retrieving realm details from '/tmp/perlscriptsinstalldemo.pl.data' Some of the input to installdemo.pl is saved here. After you are done with the demo, this may be deleted. If it gets removed, you'll have to enter everything again. Verifying DBI access to database works... DSN=dbi:Sybase:server=SOMESERVER;database=SOMEDATABASE UID=DBUSERID PWD=DBPASSWORD Connected to database: dbi:Sybase:server=SOMESERVER;database=SOMEDATABASE CREATE table 'bogosity' INSERT OK UPDATE OK SELECT OK DELETE OK DROP OK DBI access to database appears to work At this point we have verified that the Perl DBI code works. Your database userid and password can create/drop tables and put data in and get it out. Destroy the SIFTER database and re-initialize it... WARNING: destroy all data in Sifter realm 'demo' (dbtype=SYBASE)! Proceed? (y/N) y Now we are preparing to initialize the SIFTER data in your database. Running: perl/scripts/../../db/sybase/clean.sql Running: perl/scripts/../../db/sybase/schema.sql Running: perl/scripts/../../db/sybase/proc.sql Running: perl/scripts/../../db/sybase/data.sql Database for realm 'demo' has been initialized. If successful, the SQL has been run using the database userid and password and we've created a bunch of tables. Adding default attributes: Added analysis attributes Added map attributes Added enumerations SIFTER database successfully re-initialized. See log file '/tmp/installdemo.pl.sql.log' Now we've loaded some SIFTER internal data into the database. We're about ready to load some real data in. Verifying SIFTER SERVER database access works... SIFTER SERVER database access appears to work SIFTER SERVER access is fine. See log file '/tmp/installdemo.pl.checkdb.log' We have used the SIFTER realm information to access the initialized tables. Loading demo Maps into database... Will take 1-3 minutes Maps loaded into database. See log file '/tmp/installdemo.pl.adddemomaps.log' We have used SIFTER code to load the samples/Maps into the database. Loading demo Results into database... This can take 5-15 minutes Results loaded into database. See log file '/tmp/installdemo.pl.adddemoresults.log' We're going to load over sixty results into the database. As with the Maps, we will try to load the samples/Results into the database.
If you've successfully gotten this far, congratulations are in order. This says many things are working: Perl, Perl modules, the database and much of the SIFTER code. The major thing that has not been tested yet is the SIFTER code running as CGI scripts on your web server. In theory this should not be difficult to make work since you got it all working for your own userid. But web servers can be tricky, so now we install and test the web interface.
cd [whereever]/sifter-1.00 Start in SIFTER distribution directory perl -w perl/scripts/installdemo.pl -web Notice the option '-web'. It makse all the difference. If you set PERL5LIB earlier, you must do so again. You do not need database environment variables set. Retrieving realm details from '/tmp/installdemo.data' Cleaning SIFTER for install. Ignore errors. Creating Makefiles starting at '/home/YOURUSER/dev/sifter' Makefiles were created. See log file '/tmp/installdemo.serverinstall.log' We start by finding where you have installed the SIFTER code and then do 'perl Makefile.PL'. Since you have provided all the details earlier, you do not need to answer the questions. Preparing SIFTER for install... Note: Alert.java uses or overrides a deprecated API. Note: Recompile with -deprecation for details. Ignore these java compiler warnings. Note: ./ClosableFrame.java uses or overrides a deprecated API. Note: Recompile with -deprecation for details. etc. SIFTER ready for install. See log file '/tmp/installdemo.serverinstall.log' 'make server' has compiled everything and created modules with the paths you provided. Installing SIFTER... Install was successful. See log file '/tmp/installdemo.serverinstall.log' Everything is copied to the HTDOCS you specified. You should have a running SIFTER installation, but we'll check anyway :-) Check the web server... Web server is responding. All CGI activity is logged to '/tmp/installdemo.cgi.log' This just means we were able to retrieve a web page from the demo web site. This means your web url, userid and password are correct. Check CGI scripts... CGI scripts can be invoked. CGI scripts can be invoked on this web server. This means the server is configured to invoke files with a 'cgi' extension as CGI scripts. Check that CGI scripts can access all modules... CGI scripts can access all modules. Your CGI scripts can access all the Perl modules SIFTER will require. Check CGI scripts using CGI module... CGI scripts can use the CGI module. Your CGI scripts can actually use the CGI module. Check CGI scripts using DBI module... CGI scripts can use the DBI module. Your CGI scripts can use the DBI module to access the database. The same database tests we did earlier are run. Check CGI scripts using SIFTER module... CGI scripts can use the SIFTER module. We could access some SIFTER information on the web server. Your SIFTER demo site appears to work URL=https://www.YOURDOMAIN.com/sifter-demo All tests have completed for the SIFTER demo site We can't think of anything more to test. In your web browser, visit the url and select the java application (select the 'Start' button). Java is the only thing left to try out.
The SIFTER demo should work for your users. Use your web browser to access the url (e.g. https://www.YOURDOMAIN.com/sifter-demo). Select the 'Start' button and you should see a Java applet load and open a window. Try a query for results on chromosomes 6, 11, 20 or 22. Select some results and plot them and get familiar with the interface.
You should also select the other links at the top of the first page (Analysis Attr, Q_Maps and Map Attr) and see that they behave. Look at the maps and attributes and see how SIFTER defines and uses attributes.
When you are satisified you understand what's going on,
it's time to create your own project.
When the Demo Goes Wrong
SIFTER requires a number of components to work smoothly together. If this is your first installation (or first installation on this machine), you may well have a few surprises. To help assess problem areas, we provde the demo installation. You are encouraged to install the demo. It won't take long and it has the great advantage that it will do a sanity check of each of the software components. The following is a description of the checks made by the demo and if it fails, what you should investigate.
SIFTER requires a working installation of Perl. While we require version 5.6, you may well find that earlier version of 5.x will work for you. This requirement should not be a burden for modern Unix systems, as almost every system we've seen has Perl. Hope this includes you :-)
The command 'perl' should be in your PATH - again this should not be a surprise. You should also have the basic Perl modules installed (especially File::* and Getopt::Long). Again, this should be in every standard install.
The first command is 'perl installdemo.pl'. If you have Perl installed as explained above, this should compile. If this works, we have an excellent chance to test your environment.
The first check of your system is to verify all of the Perl modules needed for SIFTER are installed. This assumes they are installed in your default Perl libraries or you have PERL5LIB set. If the latter is the case, you must be sure that PERL5LIB is set for cases where SIFTER code is run (either clients loading results into the database or for the CGI scripts running on the web server).
In order to access your database directly, some database environments will require you to set an environment variable. For instance, Sybase requires that SYBASE be set. This should be no surprise for your installation.
The next check is to see if the Perl DBI code can connect to your database. This could fail because you don't have DBI installed completely or because the DSN, database userid or password is not set properly. In this same test, the Perl code will create a table, insert some data, update it, delete a row and drop the table -- all things the database userid you provide must be able to do.
Next we create and initialize all the internal SIFTER tables. Depending on your database, this requires a program that comes with your database to read the SQL and do the proper work. This might require environment variables to be set or the path to the program to be included.
Now we load a small number of maps and results into the demo database. This could fail because log files fill up, or the space for the data is inadequate.
By this point we've tested the SIFTER client code to verify the basic components are there and work. This has not been much more than a cursory sanity check. There are still plenty of environmental problems that could show up later (think of database space problems), but it's a good start.
The next phase of installdemo.pl offers to check your SIFTER web site. This can fail in several non-intuitive ways. Your web master should have done some things. We begin by checking the basics.
The first step is to 'make' the server installation. Compiling the Java code is probably the most likely thing to fail in the make. It could also fail to install because you do not have write access to the sifter-demo directory in HTDOCS (where all the web source files are saved).
After the install, we start by trying to fetch a web page. This can fail because the URL is incorrect, the protocol (http or https) was wrong, the server name was wrong or you specified HTDOCS incorrectly or the directory sifter-demo is not accessible to the web server. This can also fail because you specified a web userid and password (saved in $HOME/.sifter/user.cfg) and these values were incorrect.
The next check is that a CGI script can be invoked. This verifies the web master set up CGI scripts as was expected.. This also means that Perl is available to the web server. If this does not work, look in the error log for your web server (ask your web master).
We then check that the CGI module works for CGI scripts. If this is the case, we expect all the normal Perl modules to work. CGI failures are logged to the web servers log files. You may need to ask your web master to help figure out what's not going on if this fails.
Next we try to access the database directly. This verifies that DBI works and that the DSN and database userid and password work.
Our last CGI test is to see if the SIFTER code can access the database using the realm information you provided. This could fail because the web server does not have read access to the server directory you provided during the install. The path to this directory can be found in sifter-demo/modules/Sifter/Config/Server.pm in your HTDOCS directory. Once this works, we can be pretty confident the web server is ready.
The last test is for you to use your web browser to access the web pages. The 'Start' button should launch a Java applet. This could fail because your browser does not have Java enabled. You may want to look at the java console for information on errors (in Netscape look on the Communicator -> Tools menu).Version=$Id: demo.html,v 1.9 2002/11/22 16:41:26 tpg Exp $