RU Software Engineering - ELF - Exchange of Large Files



	Home
	Installation
	Screenshots
	Project Page

Installation

It is recommended to install ELF using a separate hard drive partition to store your uploaded files. When uploaded ELF files fill up a separate disk partition it won't cause problems with the OS. It is also recommended to specify a temporary upload directory for php (upload_tmp_dir directive in the php.ini file) that resides inside your separate hard drive partition. See the 'APACHE with MOD PHP optional MOD_SSL' PHP installation section below.

These instructions assume that your apache server is installed in /usr/local/apache and that you have a separate hard drive partition for /usr/local or /usr/local/elf. You can of course install the software wherever you like but /usr/local/apache and /usr/local/elf are used as a point of reference.

Requirements

MySQL

Apache with Mod PHP and Mod SSL(optional)

Unix OS

Perl with DBI/DBD - Perl Module

Upload - In order to upload up to 2 GB files some modification may be necessary.

By default ELF allows you to upload up to 3 files at once, however the combined size of all the uploads must be under 2 GB. Apparently, 2 GB is an http limit and/or file system limit and is probably a pain to work around. To allow 2 GB you will have to modify the php.ini(a configuration file for mod php) and change the following:

max_execution_time = 120
post_max_size = 2047M
upload_max_filesize = 2047M
upload_tmp_dir = "/mypartition/tmp"

As you can see we are 1 MB shy of 2 GB but putting 2048 would not work correctly for some reason.

PHP performs file uploads by first writing the files to a temporary directory which is located in the php.ini file under the directive upload_tmp_dir. If there are multiple files being uploaded in an html form, then each file is uploaded sequentially until the last file is done.

After all files are uploaded successfuly then your php script is able to execute and the files can be moved from the temporary directory to a directory you specify. There was an issue with the max_execution_time variable in the php.ini file not being large enough because when moving large files the script would timeout with the error message: PHP Fatal error: Maximum execution time of 30 seconds exceeded.

So, we increased the max_execution_time to 120 seconds which should hopefully resolve this issue. This time may vary depending on disk speed etc. If a user complains that his browser goes blank when uploading files then check the Apache error log files and look for execution time out errors.

MySQL

1. Install and setup MySQL, refer to www.mysql.com for more details.

Apache with mod php and mod_ssl(optional)

1. Check out the Apache website(http://httpd.apache.org/) and the 
php website(http://www.php.net/) for detailed instructions on 
installing and configuring Apache with PHP. 

2. The default ELF configuration allows for up to 2 GB of file transfers. 
When php is installed its default is much smaller. Therefore, you have 
to change the php configuration which is localed in the php.ini. This is 
usally installed in /usr/local/lib/php.ini. 

You can upload up to 3 files at once however the combined size of all 
the uploads must be under 2 GB. Apparently, 2 GB is an http limit/filesystem 
limit and is probably a pain to work around. To allow 2 GB we had to edit 
the php.ini and modify the following:

    * max_execution_time = 120
    * post_max_size = 2047M
    * upload_max_filesize = 2047M
    * upload_tmp_dir = "/ELF/tmp"

	max_execution_time - This is the maximum execution time a php script 
	can last. Some hard disks aren't fast enough to transfer the uploaded 
	files from the php temporary directory to their final resting place. 
	If you see an 'PHP Fatal error: Maximum execution time of 30 seconds 
	exceeded' in the apache error_log file then you know why.

	post_max_size - This is the maximum amount of data that a user can 
	post to an html form.
	
	upload_max_filesize = This is the maximum size that a combined file 
	upload can be.
	
	upload_tmp_dir - This should reside in the Hard Disk Partition that 
	you setup for ELF. For example, we created a the directory tmp in 
	the disk partition /ELF. This is a good idea since php will store 
	its temporary files here and not somewhere else on the system where 
	they can cause a problem for the OS.

You must restart Apache for these changes to take affect.

Perl with DBI

Perl is installed by default on most unix based systems. The Elf Daemon, 
which removes expired uploaded files and temporary accounts, uses the 
perl language. Try to determine if you have DBI installed. On some unix 
systems running:

perldoc perllocal 

will display a list of installed perl modules. If not download from
CPAN - http://search.cpan.org/search?query=DBI&mode=all and install.

ELF Layout

Basic file/directory structure of ELF: 

bin         - contains the ELF Daemon that removes uploaded files, temporary accounts,
              and database entries that expire.
db          - Upload file database where files uploaded by ELF users are stored.
gnu-elf.sql - Database schema for the ELF system.
public      - ELF web pages that need to be served from the public web space of
              your apache web server.
INSTALL     - Installation instructions.
quarantine  - Directory where viruses uploaded by ELF users are stored.
templates   - Design templates for the ELF web pages.
templates_c - Compiled design templates for the ELF web pages.

***t is recommended to install ELF on its own separate hard drive partition. When uploaded ELF files fill up the disk partition it won't cause any problems to the rest of the Operating System.

Short Version

1. Unpack ELF:

	gunzip < elf-VERSION.tar.gz | tar xvf -
	cd elf-VERSION

2. Import DB tables into MySQL 

	/usr/local/mysql/bin/mysql -u root -p < gnu-elf.sql

3. Populate the directory DB Table with your users.

4. Make necessary directories writeable by Apache webserver

	chown nobody db quarantine templates_c

5. Edit the bin/elfctl file to your environment.

6. Edit the bin/elfd file to your environment.

7. Make files executable and move necessary directories

	chmod 775 bin/elfctl bin/elfd
	mkdir /usr/local/elf
	mv db bin quarantine templates templates_c /usr/local/elf

8. Start the ELF daemon:

	/usr/local/elf/bin/elfctl start

9. Edit the configuration file: public/.cfg.php to your environment.

10. Move public directory to the web space.

	mv public /usr/local/apache/htdocs

	(or if you want a specific directory for elf)

	mv public /usr/local/apache/htdocs/elf

11. Open up a web browser to your host and test by registering yourself.

12. Secure the admin/monitor.php page with a .htaccess file.

Detailed Version

1. Unpack the elf tar ball and move into the elf directory.

gunzip < elf-VERSION.tar.gz | tar xvf -
cd elf-VERSION

2. The file: gnu-elf.sql contains the DB schema for the ELF. It will
create a database named 'gnu-elf'. Import the DB schema into mysql using 
the password you created for mysql earlier.

/usr/local/mysql/bin/mysql -u root -p < gnu-elf.sql

3. The gnu-elf database contains a table called directory. This table
allows you to specify which user can register for the ELF system. You
can populate this table with the necessary users you want for your
system. Only users in this table will be able to register. An email
address is required for the user to register. (This type of system
could change in the future to something more elegant. LDAP functionality is
in the works).

4. The db, quarantine, and templates_c directories need to be writeable by
the Apache webserver process. The Apache server is usually owned by the
user nobody but your environment may differ.

chown nobody db quarantine templates_c

5. Edit the file bin/elfctl and change the variables:

PIDFILE - The process id file for the elf daemon elfd.
ELFD - path to the elf daemon binary called elfd, located in bin directory.

6. Edit the file bin/elfd and change the 'CONFIGURATION VARIABLES' section
to fit your environment. (We are looking into a single configuration
system to work with web pages, elfctl, and elfd). 

If you experience problems with the Database connection from elfd,
the ELF Daemon specefically calls the mysql.sock from the DBI connect script:
DBI->connect("DBI:mysql:database=$db;host=localhost;mysql_socket=/var/lib/mysql/mysql.sock;",
$db_user,$db_pass);
Make sure that your mysql.sock matches the mysql.sock in this DBI connect statement. 

7. Make the bine files executable and move the bin, db, quarantine, templates, 
and templates_c directories somewhere outside the webspace. For example, if 
our apache server serves pages from the /usr/local/apache/htdocs directory 
we can place these directories in a new directory outside the web space 
such as /usr/local/elf.

chmod 775 bin/elfctl bin/elfd
mkdir /usr/local/elf
mv db bin quarantine templates templates_c /usr/local/elf

8. Try starting up your elf daemaon elfd. For example, if the bin directory
resides in /usr/local/elf/bin. 

/usr/local/elf/bin/elfctl start

9. Edit the configuration file: public/.cfg.php with values specific to your 
environment. This should take around 15 - 20 minutes. There are comments inside 
the configuration file to help you.

10. Move the public directory into your apache server's public webspace. For
example if apache servers web pages from /usr/local/apache/htdocs then:

mv public /usr/local/apache/htdocs

or if you want a specific directory for elf

mv public /usr/local/apache/htdocs/elf

11. Open up a web browser and test!

12. There is a small Admin page which shows statistics about the system, admin/monitor.php.
You will probably want to restrict this portion of the site by .htaccess files in apache
to certain users/administrators.