[[PageOutline]]

= How to integrate your own data with a local copy of FlyMine =

This page explains how to set up a local copy of the latest FlyMine release and integrate your own data.  You will be able to query the combined data in a local web application or with the query API.  The instructions here assume you need to add classes and fields to the data model.  If the data you want to load is already described by the model you can follow the standard [wiki:MirrorHowTo mirror instructions] and load your data in at the end.

= 1 Prequisites =

Ensure you have the required software installed and configured.  See: [wiki:Prerequisites prerequisites].

= 2 Get the [wiki:SVNCheckout InterMine software] =

= 3 Create PostgreSQL databases =

These correspond to database names in a flymine.properties file in the {{{.intermine}}} directory.
    {{{
createdb flymine_XX
createdb common-tgt-items
}}}
If you intend to use the web application:
    {{{
createdb userprofile-flymine_XX
}}}

= 4 Create a new datasource =

Create and modify necessary files for new source(s). 

See:  [wiki:SourceHowto creating a new source].

= 5 Create a blank database schema =

The schema will include your additions to the data model.  In flymine/dbmodel run:
    {{{
ant clean build-db
}}}
You should see table/columns corresponding to the new classes you added to the model in the production database.

= 6 Dump the metadata table from the database =

This is the table that stores an XML version of the object model.  It is needed by any client program that queries the !FlyMine.  As we have added to the model we need to preserve the copy we just created to restore after the !FlyMine data dump has been loaded:
    {{{
pg_dump -c -f updated-metadata-table -t intermine_metadata flymine_xx
}}}

= 7 Create tables =

Some tables and indexes need to be added to the production database.  You need to create these two tables because the data dump you will be loading in the next step has data for these tables.

    {{{
CREATE TABLE precompute_index (
       name character varying(255),
       "statement" bytea,
      category character varying(255)
);

CREATE TABLE tracker (
      objectid integer,
      fieldname text,
      sourcename text,
      version integer
);

CREATE INDEX tracker_objectid ON tracker(objectid);
}}}

= 8 Get the latest data =

From http://www.flymine.org/download/ (file has -data-only suffix).  Load up this dump:
    {{{
zcat dump_file | psql flymine_xx
}}}
 
= 9 Update model =

This will have overwritten the updated model we created with the previous !FlyMine model.  Now we need to load back in the updated one:
    {{{
> cat updated-metadata-table | psql flymine_xx
}}}

= 10 Check data =

Check you still have all the updated tables/columns in the database and some data.  It would be a good idea to make a new dump of the database so you don't have to repeat this procedure again.
     {{{
pg_dump -c -f updated-flymine-version-xx database_name
}}}

= 11 Analyse =

Analyse the database to update postgres' statistics.

    {{{
> psql flymine_xx
> set default_statistics_target = 500;
> analyse;
}}}

= 12 Load data =

You should now be able to load data from your own sources into this database.  In flymine integrate run:
    {{{
ant -v -Dsource=new_source integrate
}}}

= 13 Configure =

     * Copy the example [source:/trunk/intermine/doc/example/flymine.properties flymine.properties] files to your home directory
     * Edit them to include your own database and tomcat locations and usernames/passwords.

= 14 Release the web application =

In {{{flymine/webapp}}}:
    {{{
# the first time only
> ant build-db-userprofile
> ant default release-webapp
}}}
You only need to run 'build-db-userprofile' the first time you set up a webapp.  Use {{{ant remove-webapp}}} if there is already a web application deployed at the specified path:
    {{{
# to re-release the webapp
> ant default remove-webapp release-webapp
}}}