Memory-mapped files in java tutorial with example . File I/O vs Memory-Mapped Files tutorial

The following tutorial covers about what is memory mapped files and  what are the advantages and drawbacks of using Memory-Mapped Files and also covers that how to map a  file into memory with example code.
 Any files can be accessed using  

1.  Simple  File I/O
 2.  Memory-Mapped Files

Some of  the drawbacks of Simple File I/O  (Usual  read() and write()) is as follows.

 When an  application requires to read data from outside  such as   file data  on disk (outside of virtual  / process  address space)  ,  system call to usual file I/O functions (e.g., read() and write() subroutines )  , copies the file data to intermediate buffer  . Then the data is transferred  to the physical file or the process .  This  Intermediate buffering is slow and expensive which reduces the I/O performance.
 The alternative mechanishm is Memory mapped files . Memory mapped files provide a mechanism  to map the  file data  into the area of Virtual Memory (process address space) .   This enables an application, including multiple processes, to read and write  the file data directly to the memory  without performing any explicit file read or write operations on the physical file .   When we access a  part of the file which is  not  in  memory, it will be automatically paged in  by  the  OS.  Subsequent reads / writes to / from that page are treated as ordinary memory accesses .  There is no separation between modifying the data and saving it to a disk.

Some of the benefits using  Memory mapped files ( Accessing a data directly from main memory )
 1. Eliminate intermediary buffering

2. Increases I/O performance

3. More than one processes can map the same file  i.e  pages in memory can be be shared among the processes which saves memory space and  supports inter-process communications

4. supports  lazy loading i.e   the process of allocating and loading pages in main memory  must be deferred as long as possible . The page is loaded into RAM when the page is actually needed .   You don't need to have memory for the entire file.  This helps  to read  a large file with small amount of RAM

5. File data can be accessed and modified with out having to execute any explicit I/O operations  on the file.

6. Reading / Writing  large files this is often  more efficient than invoking the usual read or write methods.

      Mapping a file into memory is implemented by a FileChannel object that  is packaged with java.nio   which is available from JDK 1.4 .   The map() method of a FileChannel object  maps to  a portion or all of channel’s file  into memory  and  returns a reference to a buffer of type MappedByteBuffer .

 Syntax for the map() method is
 public abstract MappedByteBuffer map(FileChannel.MapMode mode,      long position,  long size)       throws IOException
              - Maps a region of this channel's file directly into memory.   The map() method returns a MappedByteBuffer, which is a subclass of ByteBuffer. Methods of ByteBuffer can be used with MappedByteBuffer class . 
 A region of a file may be mapped into memory in one of the following three modes:
 1. MapMode.READ_ONLY - Can not modify the resulting buffer
 2. MapMode.READ_WRITE - Can change the resulting buffer
 3. MapMode.PRIVATE  -  creates a private copies of the modified portions of the buffer  which is  not  visible to other processes hat have mapped the same file.  Modification to the resulting buffer will not be reproduced to the file

The following line of code maps the first 1024 bytes of a file into memory in Read / write mode. 

MappedByteBuffer mbb = fc.map( FileChannel.MapMode.READ_WRITE, 0, 1024 );

To map the entire file specify the start file position as zero, and the length that is mapped as the length of the file.

MappedByteBuffer mbb= fc.map(READ_WRITE, 0L,fc.size()).load();

The buffer is created with the READ_WRITE mode, which permits the buffer to be accessed or modified and maps to the entire file.  The map() method returns a reference to the MappedByteBuffer object
 Drawbacks of  Memory mapped files

                 1.  Wastage of memory for small files .  In Memory mapped files , disk block is mapped  to a page   in memory . The size of the  page  is usually  4 KB  . To map a  file  with size of 9 KB , 3 pages are allocated with total size of 12 KB in Memory.  3 KB   memory is wasted.

                 2. When a requested page is not in the main memory , page fault occurs which reduces performance.

                3. Another limitation is Maping  of file contents in memory depends on available Virtual Address Space.   32 bit OS gets a set of virtual memory addresses from 0 to 4,294,967,295 (2*32-1 = 4 GB) .
 Now let us see the code example of Maemory Mapped Files . I have written two programs  to read a large log file  using the Standard File IO and Memory-mapped I/O  and you can run the two programs to get the time taken to read the given big log files by Standard File IO and Memory-mapped I/O . Obviously , Reading large file using Memory-mapped I/O  is faster than using Standard File IO.

Using Memory-mapped I/O
  

 

import java.io.FileInputStream;

import java.io.*;

import java.nio.MappedByteBuffer;

import java.nio.channels.FileChannel;

public class MemoryMappedIO1 {

    public static void main(String[] args) {

        long tm = 0;


        FileInputStream fis = null;


        try {




           fis = new FileInputStream("CBS.log");


            int len=1024;


            byte[] buf = new byte[len];



            tm = System.currentTimeMillis();




            FileChannel fc = fis.getChannel();




            MappedByteBuffer mbb = fc.map(FileChannel.MapMode.READ_ONLY, 0, fc.size());




            while (mbb.hasRemaining()) {



 if (len>mbb.remaining())




 mbb.get(buf,0,mbb.remaining());




 else




 mbb.get(buf,0,len);



//System.out.println(new String(buf));


            }




            System.out.printf("Time to read file TestLog.log: %d ms\n", (System.currentTimeMillis()-

tm));




        }




        catch (Exception ex) {




            ex.printStackTrace(System.err);




        }




        finally {




            if (fis != null) {




                try {




                    fis.close();




                }




                catch (Exception ex) {




                }




            }




        }




    }




}



Using Standard File IO:
   

 

import java.io.BufferedReader;




import java.io.*;




public class StandardBufferedIO1 {


    public static void main(String[] args) {


        long ts, te = 0;

         InputStream in = null;


        try {


    in=new FileInputStream("CBS.log");


            ts = System.currentTimeMillis();


      byte[] buf = new byte[1024];


    int len;

    while ((len = in.read(buf)) !=-1) {


 //System.out.println(new String(buf));


      //  out.write(buf, 0, len);


    }


              te=System.currentTimeMillis();


            System.out.printf("Time taken to read log file  %d ms\n", (te-ts));


        }




        catch (Exception e) {


            e.printStackTrace(System.err);


        }


        finally {




            if (in!= null) {




                try {


                  in.close();


                }


                catch (Exception e) {e.printStackTrace(System.err);    }


            }


        }


    }


}