title: Direct Memory Access in Linux
body: <p>I'm trying to access physical memory directly for an embedded Linux project, but I'm not sure how I can best designate memory for my use.</p>
  
  <p>If I boot my device regularly, and access /dev/mem, I can easily read and write to just about anywhere I want. However, in this, I'm accessing memory that can easily be allocated to any process; which I don't want to do</p>
  
  <p>My code for /dev/mem is (all error checking, etc. removed):</p>
  
  <pre><code>mem_fd = open("/dev/mem", O_RDWR));
  mem_p = malloc(SIZE + (PAGE_SIZE - 1));
  if ((unsigned long) mem_p % PAGE_SIZE) {
      mem_p += PAGE_SIZE - ((unsigned long) mem_p % PAGE_SIZE);
  }
  mem_p = (unsigned char *) mmap(mem_p, SIZE, PROT_READ | PROT_WRITE, MAP_SHARED | MAP_FIXED, mem_fd, BASE_ADDRESS);
  </code></pre>
  
  <p>And this works. However, I'd like to be using memory that no one else will touch. I've tried limiting the amount of memory that the kernel sees by booting with mem=XXXm, and then setting BASE_ADDRESS to something above that (but below the physical memory), but it doesn't seem to be accessing the same memory consistently. </p>
  
  <p>Based on what I've seen online, I suspect I may need a kernel module (which is OK) which uses either ioremap() or remap_pfn_range() (or both???), but I have absolutely no idea how; can anyone help?</p>
  
  <p>EDIT:
  What I want is a way to always access the same physical memory (say, 1.5MB worth), and set that memory aside so that the kernel will not allocate it to any other process. </p>
  
  <p>I'm trying to reproduce a system we had in other OSes (with no memory management) whereby I could allocate a space in memory via the linker, and access it using something like </p>
  
  <pre><code>*(unsigned char *)0x12345678
  </code></pre>
  
  <p>EDIT2:
  I guess I should provide some more detail. This memory space will be used for a RAM buffer for a high performance logging solution for an embedded application. In the systems we have, there's nothing that clears or scrambles physical memory during a soft reboot. Thus, if I write a bit to a physical address X, and reboot the system, the same bit will still be set after the reboot. This has been tested on the exact same hardware running VxWorks (this logic also works nicely in Nucleus RTOS and OS20 on different platforms, FWIW). My idea was to try the same thing in Linux by addressing physical memory directly; therefore, it's essential that I get the same addresses each boot. </p>
  
  <p>I should probably clarify that this is for kernel 2.6.12 and newer. </p>
  
  <p>EDIT3:
  Here's my code, first for the kernel module, then for the userspace application.</p>
  
  <p>To use it, I boot with mem=95m, then insmod foo-module.ko, then mknod mknod /dev/foo c 32 0, then run foo-user , where it dies. Running under gdb shows that it dies at the assignment, although within gdb, I cannot dereference the address I get from mmap (although printf can) </p>
  
  <p>foo-module.c</p>
  
  <pre><code>#include &lt;linux/module.h&gt;
  #include &lt;linux/config.h&gt;
  #include &lt;linux/init.h&gt;
  #include &lt;linux/fs.h&gt;
  #include &lt;linux/mm.h&gt;
  #include &lt;asm/io.h&gt;
  
  #define VERSION_STR "1.0.0"
  #define FOO_BUFFER_SIZE (1u*1024u*1024u)
  #define FOO_BUFFER_OFFSET (95u*1024u*1024u)
  #define FOO_MAJOR 32
  #define FOO_NAME "foo"
  
  static const char *foo_version = "@(#) foo Support version " VERSION_STR " " __DATE__ " " __TIME__;
  
  static void    *pt = NULL;
  
  static int      foo_release(struct inode *inode, struct file *file);
  static int      foo_open(struct inode *inode, struct file *file);
  static int      foo_mmap(struct file *filp, struct vm_area_struct *vma);
  
  struct file_operations foo_fops = {
      .owner = THIS_MODULE,
      .llseek = NULL,
      .read = NULL,
      .write = NULL,
      .readdir = NULL,
      .poll = NULL,
      .ioctl = NULL,
      .mmap = foo_mmap,
      .open = foo_open,
      .flush = NULL,
      .release = foo_release,
      .fsync = NULL,
      .fasync = NULL,
      .lock = NULL,
      .readv = NULL,
      .writev = NULL,
  };
  
  static int __init foo_init(void)
  {
      int             i;
      printk(KERN_NOTICE "Loading foo support module\n");
      printk(KERN_INFO "Version %s\n", foo_version);
      printk(KERN_INFO "Preparing device /dev/foo\n");
      i = register_chrdev(FOO_MAJOR, FOO_NAME, &amp;foo_fops);
      if (i != 0) {
          return -EIO;
          printk(KERN_ERR "Device couldn't be registered!");
      }
      printk(KERN_NOTICE "Device ready.\n");
      printk(KERN_NOTICE "Make sure to run mknod /dev/foo c %d 0\n", FOO_MAJOR);
      printk(KERN_INFO "Allocating memory\n");
      pt = ioremap(FOO_BUFFER_OFFSET, FOO_BUFFER_SIZE);
      if (pt == NULL) {
          printk(KERN_ERR "Unable to remap memory\n");
          return 1;
      }
      printk(KERN_INFO "ioremap returned %p\n", pt);
      return 0;
  }
  static void __exit foo_exit(void)
  {
      printk(KERN_NOTICE "Unloading foo support module\n");
      unregister_chrdev(FOO_MAJOR, FOO_NAME);
      if (pt != NULL) {
          printk(KERN_INFO "Unmapping memory at %p\n", pt);
          iounmap(pt);
      } else {
          printk(KERN_WARNING "No memory to unmap!\n");
      }
      return;
  }
  static int foo_open(struct inode *inode, struct file *file)
  {
      printk("foo_open\n");
      return 0;
  }
  static int foo_release(struct inode *inode, struct file *file)
  {
      printk("foo_release\n");
      return 0;
  }
  static int foo_mmap(struct file *filp, struct vm_area_struct *vma)
  {
      int             ret;
      if (pt == NULL) {
          printk(KERN_ERR "Memory not mapped!\n");
          return -EAGAIN;
      }
      if ((vma-&gt;vm_end - vma-&gt;vm_start) != FOO_BUFFER_SIZE) {
          printk(KERN_ERR "Error: sizes don't match (buffer size = %d, requested size = %lu)\n", FOO_BUFFER_SIZE, vma-&gt;vm_end - vma-&gt;vm_start);
          return -EAGAIN;
      }
      ret = remap_pfn_range(vma, vma-&gt;vm_start, (unsigned long) pt, vma-&gt;vm_end - vma-&gt;vm_start, PAGE_SHARED);
      if (ret != 0) {
          printk(KERN_ERR "Error in calling remap_pfn_range: returned %d\n", ret);
          return -EAGAIN;
      }
      return 0;
  }
  module_init(foo_init);
  module_exit(foo_exit);
  MODULE_AUTHOR("Mike Miller");
  MODULE_LICENSE("NONE");
  MODULE_VERSION(VERSION_STR);
  MODULE_DESCRIPTION("Provides support for foo to access direct memory");
  </code></pre>
  
  <p>foo-user.c</p>
  
  <pre><code>#include &lt;sys/stat.h&gt;
  #include &lt;fcntl.h&gt;
  #include &lt;unistd.h&gt;
  #include &lt;stdio.h&gt;
  #include &lt;sys/mman.h&gt;
  
  int main(void)
  {
      int             fd;
      char           *mptr;
      fd = open("/dev/foo", O_RDWR | O_SYNC);
      if (fd == -1) {
          printf("open error...\n");
          return 1;
      }
      mptr = mmap(0, 1 * 1024 * 1024, PROT_READ | PROT_WRITE, MAP_FILE | MAP_SHARED, fd, 4096);
      printf("On start, mptr points to 0x%lX.\n",(unsigned long) mptr);
      printf("mptr points to 0x%lX. *mptr = 0x%X\n", (unsigned long) mptr, *mptr);
      mptr[0] = 'a';
      mptr[1] = 'b';
      printf("mptr points to 0x%lX. *mptr = 0x%X\n", (unsigned long) mptr, *mptr);
      close(fd);
      return 0;
  }
  </code></pre>