CSE 225 Grids and High Performance Distributed Computing

Project Teams and Planning January 22, 2003

Teams and Project Selection: January 29, 2003

Due: February 12, 2003 In class

Each of you will participate in a course project involving 2-3 students. The projects can involve a wide range of topics relevant to Grids, but must be approved in advance by the instructor. However, an appropriate project for CSE 225 will involve the use of an existing grid infrastructure and hand-on computer systems experiments involving new systems or software. In addition, past CSE225 projects have included at least one of the three following elements

· Design, implementation, and evaluation of a new grid service, algorithm, or feature,

· Design, implementation, and evaluation a grid application or algorithm, or

· a thorough performance evaluation of an existing Grid system or application.

Possible infrastructures for use in your projects include:

· the MicroGrid infrastructure from the CSAG research group (see http://www-csag.ucsd.edu/projects/grid/microgrid.html )

· the XtremeWeb or BOINC toolkits for distributed computing (see http://www.xtremweb.net and http://boinc.ssl.berkeley.edu/)

· the Entropia or Condor systems as an application (see http://www.sdsc.edu/Press/02/082002_entropia.html and http://www.cs.wisc.edu/condor

· the Globus toolkit 3.0, now in alpha release (see http://www.globus.org/ )

We have access to the source code for all of these systems, except for Condor and Entropia (we are working on this for both of them). However, it is important to understand that not all projects need involve modification to the source code. Computer system resources that will be available to support course projects include:

· a small number of Linux workstations, as part of the Active Web project (see the homework assignment),

· an Entropia deployment at SDSC,

· a larger number of Linux server (40) for work on the MicroGrid,

· some NPACI computing resources, and of course,

· any other resources you may have access to.

Possible projects include:

· (application) build an application atop the Entropia deployment at SDSC, run it extensively, and do a detailed performance evaluation

· (application) using the MicroGrid system, take an interesting Grid application, and design, model, and emulate several different alternative designs and computing resources for deployment. Clearly identify the key factors for performance and reliability in the design and computing resources.

· (MicroGrid) To simulate Grid performance effectively, network simulators must include both the grid traffic and appropriate background traffic. Based on published statistical models for internet traffic, build a set of traffic generator models which can be configured and used to run Grid modeling experiments with the MicroGrid.

· (application + MicroGrid) Build a Grid application which has similar capabilities to Shoch’s Worm, and explore its dynamic behavior (growth and shrinkage) on the MicroGrid simulator

• (application + Globus + Condor-G) Condor-G does advanced job scheduling and submission (compared to globusrun); it is also a fault-tolerant system with checkpoint/migration mechanisms.  Build an application which uses Condor-G to run across either a Globus testbed or on a virtual grid topology over MicroGrid. Inject errors and resource losses to characterize the overhead of checkpointing and efficiency of recovery.

· (Desktop Grids/Parallel) Using the XtremeWeb or Boinc toolkits on a small number of systems, and develop a resource management and scheduling extension which schedules groups of machines to run “parallel” jobs (such as MPI or PVM), arranges for parallel communication, and detects failures terminates processes in response. Based on a small testbed, demonstrate this functionality on the NAS parallel benchmark applications.

· (Desktop Grids/Data-intensive) Using the XtremeWeb or Boinc toolkits on a small number of systems, and develop a resource management and extension which schedules tasks to machines based on data location, and replicates data on demand when there is no clear good location for scheduling. Explore the performance of the system based on a range of scheduling heuristics. Demonstrate this functionality using BLAST on a segmented database (or other suitable program).

· (OGSA) Building on homework #2, design and implement an OGSA grid service front end to any of the following systems: Condor, Entropia, XtremeWeb, or Boinc. Demonstrate this system by using UNICORE (or other OGSA client systems) to submit and run some application thru this interface.

· (OGSA) Building on homework #2, design and implement an application service (e.g. something like a portal) as an OGSA grid service which automatically selects from several resources based on load factors and demonstrate its functionality.

For other topics, please develop a brief description and discuss with Professor Chien to get approval in advance.

What to turn in for your Project Plan

Organize into groups of 2-3 students and select a project

• Email Professor Chien collectively with your preference for the project topic January 29, 2003.
• Develop a project plan based on
• Obtaining the relevant software
• Exploring current functionality and needed extensions
• Exploring availability of system resources
• Software design
• Develop an implementation and testing plan for that design
• Write this up and submit it
• Your project should have significant (and verifiable milestones for week 2, 3, and 4)
• Think carefully about your design and be ambitious, but realistic about the effort involved.
• You have approximately 5 weeks to do the project from the time the project plan is submitted