Server Development Setup
From Mugshot Developer Wiki
To work on the Mugshot server, you need to install a set of dependencies and development tools, and then get a debug/test instance of the server running on your workstation.
The instructions here tend to become out of date; if you go through them and find a mistake or encounter a problem, PLEASE ask about it, and update this page so other people won't have the same issue.
See also Server Code Overview, Windows Client Development Setup and Linux Client Development Setup.
Prerequisites
- A recent Linux system.
- The production mugshot.org server runs on Red Hat Enterprise Linux 4.
- We typically use a recent Fedora Core for our workstations, Enterprise Linux 4 would also work for maximum similarity to production.
- When installing, choose groups such as Server, Web + MySQL + Java Development to save "yum install" work later on
- OpenJDK 6
- On Fedora 8, you can do this: "yum install java-1.7.0-icedtea-devel", then "yum --enablerepo=development install java-1.6.0-openjdk-devel" Be sure to install the 1.7.0-icedtea first to pull in the Fedora 8 dependencies.
- We have a specific JBoss version, in RPM format, which you should install.
- MySQL (yum install mysql-server)
- ant (yum install ant ant-junit junit)
- Eclipse
- You can build and run the server without Eclipse, and do simple patches, but to really work on it Eclipse is effectively required
- We usually install Subclipse, and the WDT (web development tools) add-ons to Eclipse, though they are optional
- Memory. Eclipse and JBoss between them are easily going to eat a gigabyte of RAM, and that's about the minimum to run both without feeling some pain. You can run them on two different machines, though, if that helps (export the source tree via NFS or Samba, and have Eclipse access it from another system).
Overview
There's more than one way to do this. Your goal is an edit-compile-test-debug cycle.
If you normally run Windows, you may want to run Eclipse on Windows, but have the source tree on a file share hosted by Linux (the Linux system could be physical or VMWare/Xen). Then log in to Linux to compile-test-debug. If you have a Linux workstation, you can run Eclipse under Linux.
The flow is:
- Edit and save the .jsp, .java, .js, or whatever in Eclipse
- Eclipse will be compiling as you do this and tell you about syntax/compilation errors
- But... the actual build uses Ant, the Eclipse build is just for code completion and syntax checking.
- Run a build either for all of the server/ directory or the subdir you modified
- Run our "super" script in "reload" mode to deploy the new build to your debug instance of the server
- Test your changes
- If they break, use the log files to try and figure it out
- And round and round...
Get Eclipse
- Get the latest stable Eclipse from the Eclipse web site.
- Get Subclipse from its main site, they have instructions for installing it from inside Eclipse.
- (Optional, for .jsp/javascript/html editor modes) get the "WDT" tools from the Eclipse site.
You can use the Fedora packages for Eclipse, but you will need to run Eclipse with the Sun JDK in order to get Java 5 support, at least as of now. It's probably simpler to install the Eclipse from eclipse.org.
Check Out Source Code
In Eclipse, open the "SVN Repository Exploring" perspective. This is part of Subclipse that you installed earlier.
Enter the Mugshot subversion repository information.
Then browse to the module "dumbhippo/trunk" and do "Check Out As Project"
IMPORTANT you need "Check Out As Project" not regular "Check Out" so Eclipse will import the project files.
By default the project will be checked out to the 'trunk' directory in your workspace, you may want to choose a different name.
Get it Building Inside Eclipse
To get things building inside Eclipse, you'll have to set two "classpath variables" and one "linked resource" variable.
First, you need to tell Eclipse where to find all the jar files we depend on. Most of them are in /opt/jboss (from the JBoss RPM), and you can find them all at once by setting the JBOSS_HOME classpath variable in Eclipse.
To set JBOSS_HOME as a Classpath Variable within Eclipse, Select Window->Preferences, then from the tree on the left select Java->BuildPath->Classpath Variable and select NEW variable. Here you can add JBOSS_HOME and set it to the root of the jboss installation.
Second, you need to tell Eclipse where to find the jar files that come with our project. These are in trunk/lib dir you checked out the Eclipse project from Subversion normally. Set the variable HIPPO_TRUNK_LIB in the same way you set JBOSS_HOME, this time pointing to the trunk/lib directory in the project.
If you have the project on a network share, it's a good idea to check out trunk/lib separately on a local disk, and point HIPPO_TRUNK_LIB at the local copy. This will make Eclipse a lot faster.
Third, you need to point Eclipse at the right output directory. Choose Window->Preferences, then from the tree on the left select General->Workspace->Linked Resources. Here you can define a variable called ECLIPSE_BUILD_DIR.
The ECLIPSE_BUILD_DIR is where Eclipse will stick .class files. These are never run; they are only used for syntax checking and code completion and so forth. Thus, put them on a local disk or somewhere fast. If you put this directory on a network share or even a different virtual machine, Eclipse can be painfully sluggish.
Eclipse has no way we know of to set defaults for variables, if it did we would set them (JBOSS_HOME=/opt/jboss, HIPPO_TRUNK_LIB=trunk/lib, and ECLIPSE_BUILD_DIR=trunk/bin). Since there are no defaults, unfortunately you have to do this manually.
Note that the trunk/ directory is your project checkout directory, so if you named it something else, use that name.
If you check out multiple branches of the project, note that these three variables are global for all checkouts. You'll need to modify one of your checkouts to use different variable names. This may be done for you already in Subversion for some branches.
If things don't build in Eclipse after setting all these variables, try running 'ant' once outside of Eclipse; it generates a couple of files Eclipse may need.
Warnings Configuration
Some of the imported source triggers a lot of Eclipse compiler warnings (> 600 at this time); we don't want to change all of the imported code, but neither do we want to disable the warnings. The way to solve this to add a filter to the Eclipse warnings. Click on the multiple-arrows icon in the top right of the errors/warnings console. Click "On working set", then Select. Click "New", then "Java". Then pick server/src and wildfire/src/plugins/hippo/src/java.
Spell Checking
Tag, jsp, and css files don't land themselves well to spell checking. You can disable spell checking by opening the Window->Preferences dialog, and deselecting it in the General->Editors->Text Editors->Spelling tab.
Get it Building for Deployment
Next you need a build that's usable for testing; the Eclipse build is just compiling Java bytecode to check for syntax errors, but it's not creating deployable packages.
Each subdirectory of trunk/ (server, wildfire, imbot) has its own build setup using Ant. There's also a toplevel trunk/build.xml which has convenience targets to recursively build the other three programs.
The first thing you will need to do is create the deployment directory:
mkdir ~/dhdeploy
You'll only need to do this once. You can also configure the directory if you're so inclined.
To build everything this should work:
ant
Note for Fedora 9 users : Install TurboGears package before running below super/super reload all script
- TurboGears (yum install TurboGears)
You can then use the Super scripts to create or update a deployment; the default configuration will deploy to ~/dhdeploy. To redeploy you would use commands like:
super/super reload all super/super reload jboss
The "reload" command does the minimum to redeploy your current build; it might have to restart JBoss or might just push a new version of the .ear file for JBoss to dynamically reload.
You can restart a server but not push the latest build to it:
super/super restart jive
There's also an ant target which builds everything then reloads everything:
ant reload
You'll notice that when reloading jboss, the reload command returns long before JBoss has fully unpacked the application; you can't try your changes until JBoss finishes. You can use the super watch command to watch the logs and see when the app finishes loading:
super/super watch jboss
Or you can manually look at ~/dhdeploy/jboss/log/server.log, or you can just bang on the reload button in your browser until JBoss starts responding. ;-)
Accessing Your Debug Deployment
When you get it all started up, you can go to http://localhost.localdomain:8080 for the web site and the XMPP server is on localhost.localdomain:21020 (assuming a default Super configuration and browser on the same machine as the server).
The hostname you use to access the server in your browser MUST match the serverHost parameter in your ~/.super.conf (or the default serverHost in super/base.conf). Most people set serverHost to "localinstance.mugshot.org" and configure their /etc/hosts (on Linux) or Windows\System32\drivers\etc\hosts on Windows to point localinstance.mugshot.org at the server's IP address.
Note that each instance really needs a different hostname, not just a different port; if you configure two instances with the same hostname and different ports, weird stuff can happen - one problem is that it's not predictable enough what browsers do with cookies in this case.
The Mugshot clients support three "instances" (debug, dogfood, and production). You can launch a client on the debug or dogfood instance with the --debug or --dogfood command line options. See Windows Client Development Setup and Linux Client Development Setup for details on this.
There's a chicken and egg problem with a fresh server - you need an invitation to create an account, but there's nobody on the system to invite you. In .super.conf you can set disableAuthentication=true, which means you can log in as any email address or AIM screen name, and the account will be autocreated and any password will be accepted for any account. Obviously, don't use this on public server instances.
Sample .super.conf for a Debug Deployment
You need to place this .super.conf file in your home directory.
<?xml version="1.0" encoding="UTF-8"?>
<superconf>
<parameter name="serverHost">localinstance.mugshot.org</parameter>
<parameter name="serverHostGnome">localinstance-online.gnome.org</parameter>
<!-- IP address of your computer -->
<parameter name="bindHost">YourIPAddressHere</parameter>
<parameter name="serverFeatures">pollingTask</parameter>
<parameter name="disableAuthentication">true</parameter>
<parameter name="adminConsole">true</parameter>
<parameter name="webVersion">3</parameter>
<!-- see the comment in base.conf for what this should be -->
<parameter name="multicastAddress">239.255.38.38</parameter>
<!-- Invitations for a regular new user -->
<parameter name="newUserInvitationCount">10</parameter>
<!-- Other parameters you'll need if you are testing specific features.
You will likely need to create your own API keys for each, but let
us know if you have questions about any of them. -->
<!--
<parameter name="imbotScreenName">ScreenNameHere</parameter>
<service name="imbot">
<parameter name="imbotPassword">PasswordHere</parameter>
</service>
<parameter name="aimPresenceKey">AimPresenceKeyHere</parameter>
<parameter name="amazonAccessKeyId">AmazonAccessKeyIdHere</parameter>
<parameter name="amazonAssociateTag">AmazonAssociateTagHere</parameter>
<parameter name="flickrApiId">FlickrApiIdHere</parameter>
<parameter name="facebookApiKey">FacebookApiKeyHere</parameter>
<parameter name="facebookSecret">FacebookSecretHere</parameter>
-->
</superconf>
You also need to add the following line in your /etc/hosts if you are on Linux (and something similar in Windows\System32\drivers\etc\hosts if you are on Windows)
YourIPAddressHere localinstance.mugshot.org localinstance-online.gnome.org
Clustering
Server Clustering is the page dedicated to a clustered setup.
Debugging and Administering
We have a page dedicated to debugging and in general administering the server: Server Admin
JBoss Runs Out of Memory
Every so many reloads, JBoss will just stop working; when you look in the logs the exception will be something like "Out of PermGen Space." When this happens you have to find the JBoss process and "kill -9 PID", then "super/super restart jboss". If you find the memory leak and fix it we'll love you. The leak so far seems to be caused by reloads, we don't see a leak on the production servers over time, only on debug instances after tons of reloads. Presumably we somehow leak some classes from our .ear or something like that.
Eclipse Tips
- Enable the warning for failure to use @Override, since we require use of @Override in our code
- Filter the pattern "*.jar" out of the package explorer, otherwise the jar files create a bunch of clutter
See also the generic Eclipse_Tips page.
Server Test Suite
There's a black box test suite for the server called Hungry; it has its own dedicated page describing how to use it.
Applications database
Get the dappinfo files from /srv/cluster on the cluster, then in trunk/tools there's a dappinfo tool that you run as: dappinfo upload *.dappinfo dappinfo --server=localinstance.mugshot.org upload *.dappinfo, that is You'll have to be logged into an account with admin privileges
