Remote Direct Memory Access Over IP

doormat writes "Accessing another computer's memory over the internet? It might not be that far off. Sounds like a great tool for clustering, especially considering that the new motherboards have gigabit ethernet and a link directly to the northbridge/MCH."

Comments

Also

[madcoder47]

Not to mention easy access to sensitive information in emails, documents, and PIMs that the user currently is running and are resident in memory.

Re:Also

[Urkki]

As it is, even programs running on the *same* computer can't access each others memory. I don't see how they could create a network-shared memory that would unintentionally get around this.

haha... outlook worm writers will have a field day

[sisukapalli1]

Seriously though... this is where Scott McNealy's vision of "The Network is the Computer" comes even closer to reality.

S

Re:rdma?

[astrashe]

You hit the nail on the head -- the security implications of this are staggering.

And doesn't tcp/ip involve a lot of overhead for memory access?

I smell a hotfix...

[fejrskov]

> Microsoft ultimately is expected to support RDMA
> over TCP/IP in all versions of Windows

Can you see it coming? The ultimate Windows root exploit!! Hmm... I guess someone has to go tell them. Othervise they won't notice it until it's too late...

Seriously, how do you dare to enable this kind of access?!?

Re:I smell a hotfix...

[Flakeloaf]

They've done it - a way has been found to make the Windows environment less secure. I can just see the sales pitch now.

"With Windows CX, your computer will have the latest in Remote Memory Management. Share your system's power with another Windows PC for added performance. Trusted applications will automatically control your memory remotely, saving you the trouble of worrying about the wrong programs using your PC."

(Which, in the usual MS doublespeak, means Bill's trusted computer can bork warezed versions of anything by remote control without having to resort to TCPA.)

Bah, old stuff

[Erich]

There's lots of research about network shared memory for use in various things.

It's very interesting that using memory over the network is very much the same problem as cache coherency amongst processors. If you have multiple processors, you don't want to have to go out to the slow memory when the data you want is in your neighbors cache... so perhaps you grab it from the neighbor's cache.

Similarly, if you have many computers on a network, and you are out of RAM, and your nighbor has extra RAM, you don't want to page out to your slow disk when you can use your neighbor's memory.

NUMA machines are somewhere in between these two scenarios.

There are lots of problems: networks aren't very reliable, there's lots of network balancing issues, etc. But it's certainly interesting research, and can be useful for the right application, I guess.

Disk is slow, though... memory access time is measured in ns, disk access time is in ms... that's a 1,000,000x difference. So paging to someone else's RAM over the network can be more efficient.

I don't have any good papers handy, but I'm sure you can google for some.

I can already do this...

[gweihir]

1. ssh root@remote-machine
2. read from and write to /proc/kcore in remote-machine

So where is the use of that? And shared memory emulation over a network is also a decades old technology.

Security Implications

[Acidic_Diarrhea]

Yes, there are security implications but here's the point: there are security implications for a lot of applications written for your computer. These obviously have to be taken into account but just because there are risks involved doesn't mean that attempting an implementation should entirely be avoided. I mean, there are security implications for running a web server (especially an out of date one with exploits all over the place); does that mean no one should run a web server?

I think that shared memory across a network is doable but, like all initial attempts, bugs will exist. But the benefits of having shared memory like this outweight the drawbacks of having a hard problem to solve.

Re:uh....

[girl_geek_antinomy]

so, computer 1 asks for a memory address from computer 2, and can then read or write to it by sending back a command.

Operating system mediated memory protection might be an issue here... Sane operating systems at least check to see whether Application 1 actually owns the bit of memory it's trying to read/write before letting it chew over memory that actually belongs to Application 2. Just letting some application read and write any memory is a recipe for disaster that sensible OSes have avoided for a long time...

You're not doing the same thing

[Acidic_Diarrhea]

From the article:"It helps reduce latency in data transfers between systems by directly placing data from one system's main memory to another's without the need for extensive buffering or CPU intervention."

The approach you describe relies on CPU intervention on both ends of the connection. The article describes an approach that is much closer to the actual hardware than simply opening a ssh connection. I hope this clears the issue up for you!

Worst possible technique for distibuted systems.

[johnjaydk]

Now shared memory might be an incredible neat solutions, in theory. In a multi-cpu box with a shared databus the system holds water but not in a hetrogene, lose coupled system.

The amount of book-keeping required to keep this thing going makes it a non-starter. And as for scale'ing. Forget it.

The sad truth is that it's common knowledge that this is the least efficient principle for distributed systems. This technique is usually the fall-back position if nothing else works.

What this is REALLY for

[calica]

First off, this is not a network shared memory scheme. RDMA could be used to implement one very efficently though.

It will not allow arbitary access to your memory space. In fact, it would prevent a great number of buffer overflow exploits

The best analogy is the difference between PIO and UDMA modes of your IDE devices (or any device). This is all about offloading work from your CPU. It is moving the TCP/IP stack from the kernel to the network card for a very specific protocol.

Here's how RDMA would work layered over (under?) HTTP.
- browser creates GET request in a buffer
- browser tells NIC address of buffer and who to send it to.
- NIC does a DMA transfer to get buffer. OS not involved
- NIC opens RDMA connection to webserver
- server NIC has already been told by the webserver what buffer it should put incoming data
- webserver unblocks once data in buffer and parses it.
- webserver creates HTML page in second buffer.
- webserver tells server NIC to do a RDMA transfer from buffer to browser host
- client NIC takes data and puts it in browser buffer
- browser unblocks parse HTML and displays it.

All of this with minimal interaction with the TCP/IP stack. RDMA just allows you to move a buffer from one machine to another without alot of memory copying in the TCPIP stack.

In fact, the RDMA protocol could be emulated completely in software. It would probably have a small overhead verses current techniques but would still be useful. Just imagine real RDMA on the server and emulated RDMA on the clients (cheaper NIC). The server has less overhead and most clients have cycles to spare!

Just one problem...

[NerveGas]

Sounds like a great tool for clustering, especially considering that the new motherboards have gigabit ethernet and a link directly to the northbridge/MCH.

There's just one problem with that... ethernet (even GigE) is *not* a good connection for clustering. Sure, the bandwidth is semi-decent, but the *latency* is the killer. Instead of a processer waiting a number of nanoseconds for memory (as with local memory), it'll end up waiting as much as milliseconds. That may not sound like much, but from nano to micro you jump seven orders of mangitude!

steve