The NPAC Visible Human Viewer

Yuh-Jye Chang
Northeast Parallel Architectures Center, Syracuse University
B29-8 Slocum Height, Syracuse, NY 13210
yjchang@npac.syr.edu
http://www.npac.syr.edu/users/yjchang/

Abstract:
The NPAC Visible Human Viewer is a user friendly interface with easy to understand descriptions and a user guide online. The user can explore every slice of the Visible Human data set from three different viewpoints (axial, coronal, and sagittal), preview the sample slices and then manipulate the full size images by popup image windows.

This paper discuss about the Visible Human project, the Visible Human data set, data set process, and the most important - put it online by using JAVA programming language.

Keywords:
axial, coronal, sagittal, Java, applet, RAW, PPM, Anatomy, CT, MRI.

Table of Contents

Java Applet: The NPAC Visible Human Viewer
About the Visible Human Project
About Visible Human Data Set
Download and Store Data Set from Internet
Prepare the Data Set
Crop and Adjust the Image
Selecting the Right Software Package for Rendering
Determine the Right Programming Language
Programming Design in JAVA
Results
Conclusions
References
Author Biographies

Java Applet:

The NPAC Visible Human Viewer

About the Visible Human Project

The National Library of Medicine has undertaken to provide a set of digitized images of the human body for use in education and research. The Visible Human Project will initially create a digital image data set of a complete human male and female cadaver, with digitized anatomical photographs, as well as magnetic resonance imaging (MRI) and computer tomography (CT) data.

The Northeast Parallel Architectures Center (NPAC) at Syracuse University has obtained a copy of the Visible Human data set and is working on processing the raw data to produce two and three dimensional images that can be accessed over the Internet for educational purposes.

Currently the images are only available as two dimensional slices, via The NPAC Visible Human Viewer, an interactive graphical interface written in Java. The viewer is only accessible using Web browsers that support Java applets, such as Netscape 2.0.

About Visible Human Data Set

The Visible Human Data Set consists of Magnetic Resonance Interferometry (MRI), Computerized Tomography (CT) and anatomical images. Axial MRI images of the head and neck and longitudinal sections of the rest of the body were obtained at 4 mm intervals. The MRI images are 256 pixel by 256 pixel resolution. Each pixel has 12 bits of grey tone resolution. The CT data consists of axial CT scans of the entire body taken at 1 mm intervals at a resolution of 512 pixels by 512 pixels where each pixel is made up of 12 bits of grey tone. The axial anatomical images are 2048 pixels by 1216 pixels where each pixel is defined by 24 bits of color, about 7.5 megabytes. The anatomical cross-sections are also at 1 mm intervals and coincide with the CT axial images. There are 1871 cross-sections for each mode, CT and anatomy. The complete male data set is 15 gigabytes in size.

Data store in RAW input format as follow:
Type Header Width Height Channels Interlaced
Anatomy 0 2040 1216 3 no
CT 3416 512 512 2 yes
MRI 7900 256 256 2 yes

Download and Store Data Set from Internet

Downloading 15 gigabytes data from the Internet is not an easy job. The files we download from vhp.nlm.nih.gov are Z compressed. After download and uncompressed, image files are store in RAW format that is not support in most existing image applications. we had to figure out the RAW format and write a program that can convert RAW format to PPM format (PPM, or portable pixmap, is very popular in Unix Operating System).

The compressed files save a lot of space, but they are not suitable for real-time rendering and display. So we still need 15 gigabytes to store the whole data set.

After downloading, converting and checking some slices, we found that we can crop the image to smaller size and save some space. After I go through all the 1878 slices and decide the crop arguments (including width, height, x-offset, and y-offset), we can fit the data into about 6 gigabytes space.

Prepare the Data Set

When all the data are ready, we took a closer look at the images and found some problems:
  1. Some images are totaly black. This is caused by some error operations by NLM and can not be recovered. So we just leave those files blank.
  2. Some Images are bad. We believe it is also caused by the problems in the NLM data collection.
  3. Some images are shifted and not aligned with the adjacent image. I already spend some time to determine the actual offset and made some corrections.
  4. In order to fix the body, they apply semi-transparent blue gelatin to surround the body and make the image full of blue color outside the body. We have written some programs to eliminate it.
  5. The people in NLM use blue color to fill in the holes in the images. (body would have some holes in some sections). The same program we use for the previous problem can be applied to this one, too.

Crop and Adjust the Images

In order to save space and clean up the images, we crop and adjust the images as following:

Slice Range Dimensions(pixels) Crop Offsets Adjust Offsets
1001-1219 648x730 (700,250) (13,0)
1120-1230 1128x730 (460,200) (13,0)
1131-1239 1128x730 (460,200) (3,0)
1240-1252 1728x790 (160,100) (3,0)
1253-1360 1728x790 (160,100) (13,0)
1261-1390 1728x790 (160,100) (0,0)
1291-1416 1728x790 (160,100) (13,0)
1417-1499 1728x870 (160,100) (0,0)
1500-1599 1728x910 (160,100) (0,0)
1600-1699 1728x940 (160,100) (0,0)
1700-1749 1572x936 (238,100) (0,0)
1750-1799 1408x970 (320,100) (0,0)
1800-1809 1268x970 (390,100) (0,0)
1810-1839 1288x1000 (380,100) (0,0)
1840-1874 1328x970 (360,100) (0,0)
1875-1899 1308x944 (370,100) (0,0)
1900-1924 1308x880 (350,100) (0,0)
1925-1999 1348x800 (350,100) (0,0)
2000-2099 1348x794 (350,100) (0,0)
2100-2199 1348x694 (350,144) (0,0)
2200-2299 1232x584 (408,200) (0,0)
2300-2399 1288x614 (380,126) (0,0)
2400-2499 1328x500 (360,110) (0,0)
2500-2599 1328x436 (360,132) (0,0)
2600-2699 1308x366 (370,184) (0,0)
2700-2774 1328x520 (360,152) (0,0)
2775-2819 1348x730 (350,152) (0,0)
2820-2878 1348x730 (350,220) (0,0)

Selecting the Right Software Package for Rendering

In the beginning stage, we want to search for a software package that can handle true-color image, large data set, easy to use, multi-platform support, 3d volume rendering, client/server, support Web access and cost under $10K. All of those packages such as NIH image, 3dviewnix, Khoros 2.0, VROOM, Wavefront, ..., can not match all the requirements we make. Most of them fail on two or three requirements. In the current stage, we just provide the 2D operations in the interface. We may have to write the 3D rendering procedure without using a pre-existing software package in the future.

Determine the Right Programming Language

At first, the X-window program seemed to be the best choice for this project. It is industry standard and running on any X-terminal remotely. The client and server mechanism works well. But, not every one is running on the Unix operation system, they may only have a MAC or PC at home with a modem connection. Obviously, most of them will have a terminal emulator without X-terminal compatibility. From this point of view, we have to write a program that can fit into different platforms. Even worse, the user would have to down load a program binary suitable for their platform and even the huge data set. That make the whole thing become very complex and cumbersome.

Sun Microsystem, Inc. have announced the JAVA final release version. It has a very strong Internet support and cross-platform design. Netscape already released the 2.0 release that supports JAVA. By using JAVA, we can write just one program, put it on a Web site, and everybody can get access to the service remotely by using a JAVA-enabled Web browser no matter what his operating system is. For this application, JAVA was proved to be the right choice.

Programming Design in JAVA

I personally am a C++ Microsoft Windows application programmer with 5 years experience. JAVA is almost a C++ language that has some slight differences such as no pointer support, no multi class inheritance, support internet access protocol, ... and so on. By reading sample programs and the JAVA newsgroup, I picked up this language in a short time.

The NPAC Visible Human Viewer is an JAVA applet, that contain 12 files: 
VisibleHuman.java  -- the Main Applet
ImageFrame.java    -- the Popup Image Frame
ImageSelector.java -- Image Object with a Selection Line
ImageCanvas.java   -- Image Process and Display Object
Scroller.java      -- Motif Look Scrollbar
Button3D.java      -- Motif Look 3D Button
Panel3D.java       -- Panel with 3D Outlook
Label3D.java       -- Label with 3D Outlook
Canvas3D.java      -- Canvas with 3D Outlook
Thumb.java         -- Thumb in Scrollbar
Setting.java       -- Global Setting Interface
Delay.java         -- Delay Interrupt Object
Use the AWT component provided by JAVA is simple. But there are some problems arise when using AWT components:
  • Component will have different default size in different platforms.
  • Component will have different look and feel in different platforms.
  • Some special functions will hard to be achived. eg. Sometimes, you want the action take place when button is pressed. Sometimes, you want it take place after button is released. Sometimes, you want a button has auto-repeat function, .....

I personally believe UI (user interface) is the most important element in commercial application design. If you are planning to write some serious application on JAVA, you will find the components provided in JAVA AWT is not enough. So I spend a lot of time on UI layout , process flow, and component redesign. It makes this program more complex than it should be, but people will like it more.

The Class Hierarchy in NPAC Visible Human Viewer:

Object --- Delay
Applet --- VisibleHuman
Frame  --- ImageFrame
Panel  --- Panel3D --- Scroller
Canvas --- Canvas3D -+- Label3D --- Button3D
                     |
                     +- ImageCanvas --- ImageSelector
                     |
                     +- Thumb
In Panel3D class, I put in the concave/convex drawing utility methods. So the child classes will take advantage of these methods and make the concave/convex's color and style follow the same standard in different components.

The Component Hierarchy:

VisibleHuman -+- board ----+- view[3] --+- logo (ImageCanvas)
              |  (Panel3D) |  (Panel)   |  
              |            |            +- npac (Button3D)
              |            |               
              |            +- view[0] --+- select[0] (ImageSelector)
              |            |  (Panel)   |  
              |            |            +- tagBtn[0] (Button3D)
              |            |               ("Axial")
              |            +- view[1] --+- select[1] (ImageSelector)
              |            |  (Panel)   |  
              |            |            +- tagBtn[1] (Button3D)
              |            |               ("Sagittal")
              |            +- view[2] --+- select[2] (ImageSelector)
              |               (Panel)   |  
              |                         +- tagBtn[2] (Button3D)
              |                            ("Coronal")
              +- panel ----+- resolutions -+- tagBtn[3] (Button3D)
                 (Panel)   |  (Panel3D)    |  ("Low")
                           |               +- tagBtn[4] (Button3D)
                           |               |  ("Medium")
                           |               +- tagBtn[5] (Button3D)
                           |                  ("High")
                           +- controls -+- adjust ---+- left (Button3D)
                              (Panel3D) |  (Panel3D) |  (triangle)
                                        |            +- right (Button3D)
                                        |               (triangle)
                                        +- slice (Label3D)
                                        |  (current slice number)
                                        +- load (Button3D)

ImageFrame Component Hierarchy:

ImageFrame -+- picture ---+- canvas (ImageCanvas)
            |  (Panel)    |  (Center)
            |             +- hScroll (Scroller)
            |             |  (South, HORIZONTAL)
            |             +- vScroll (Scroller)
            |                (East, VERTICAL)
            +- controls --+- Quit (Button3D)
               (Panel3D)  |  ("Quit")
                          +- Download (Button3D)
                          |  ("Download")
                          +- Apply (Button3D)
                          |  ("Apply")
                          +- Org (Button3D)
                          |  ("1:1")
                          +- lbScale (Panel3D)
                          |  (current scale)
                          +- sbScale (Scrller)
                             (HORIZONTAL)

Scroller Component Hierarchy:

Scroller ----+- up (Button3D)
(HORIZONTAL) |  (West, triangle)
             +- down (Button3D)
             |  (East, triangle)
             +- thumb (Thumb)
                (Center)

Scroller ----+- up (Button3D)
(VERTICAL)   |  (North, triangle)
             +- down (Button3D)
             |  (South, triangle)
             +- thumb (Thumb)
                (Center)

Results

The result is a user friendly interface with easy to understand descriptions and a user guide online. The user can explore every slice of the Visible Human data set from three different viewpoints (axial, coronal, and sagittal), preview the sample slices and then manipulate the full size images by popup image windows. We plan to add MRI and CT data sets and 3D rendering interface. When the female data set is available, we will support it in a short time.

Conclusions

Recently, Internet access has become a very hot issue. Inside the Internet, there are huge man power, machines, opportunity, information, and resources. I believe tele-education will become one aspect of those very popular and important new technologies in the future.

References

Author Biographies

Yuh-Jye Chang Ph.D student at Syracuse University since Aug 1994. I am now working on The NPAC Visible Human visualization project under the advise of Dr. Paul Coddington. My first public Java applet The NPAC Visible Human Viewer was rated the No.1 and top 1% applet by JARS in Dec 1995. I am also the share holder and project leader of Newtek Corp. in Taipei, Taiwan. I initiated and completed several commercial projects using C++ on Chinese TrueType font related applications.