Apple was definitely behind the power, performance curve," said Nathan Brookwood, principal analyst at Insight 64. The PowerPC processor that emerged from that earlier pairing changed that
PowerPC was pushed by the AIM alliance: Apple, IBM, Motorola. The latter two developed and produced chips. Apple had some input. The goal was an ISA that made it easy to emulate both m68k and i386.
Yup, it's on GitHub in two repositories, one for LLVM and one for clang. We've been pushing the fixes for MIPS IV upstream, but there are some changes to the mid-level optimisers to make them aware that not all pointers are integers and some extensions to the platform-independent code generators for the same.
We've upstreamed all of the FreeBSD changes required for the base processor (BERI) and the development boards, so FreeBSD 10 will run out of the box on CHERI (it just won't expose any of the capability features). The CHERI-enabled version of FreeBSD is also on GitHub.
The code for the processor is just in a tarball at the moment. We're going to move it to some sensible public revision control system once we've worked out how much of our internal svn history wants to be included in the public release.
This is an interesting design. Have you thought about maybe posting seL4 to it once that's open-sourced in a couple of weeks?/akr
Yes, we're looking forward to the seL4 release. Somewhat cautiously, as there's no mention of a license (if it's GPL'd, we probably won't put any effort into it). In CHERI, we have store-capability and load-capability permissions on both capabilities and pages. To implement the seL4 model (where capabilities are stored in special pages), we'd only need to use the TLB-grandularity protections, but it would be more interesting to provide fine-grained capability storage, which would reduce a bit of the difficulty in programming for seL4.
We're also interested in the impact on the proofs in seL4. For example, seL4 currently assumes that the TLB is correct as an axiom (read the errata sheet for any production CPU to see how valid that is). With CHERI, we have some proofs extending down into the ISA and into the implementation and it would be interesting to see how these could be coupled to have a proven-secure systems. The folks at UT Austin and Centaur are (independently of us, but we talk to them periodically) working hard on proving that properties of higher-level HDLs are mapped correctly to gate layouts, so in a few years it may be possible to have a microkernel with security properties proven correct right down to the gate level.
Ilya Bakulin is currently working on porting FreeBSD to run atop L4 and is interested in extending this port to run on seL4. That would also be an interesting approach when combined with CHERI, allowing a full FreeBSD to run with some critical services moved out and into a tiny TCB.
We haven't specifically looked at key storage, but we are looking at potential ways of extending the approach to heterogeneous multicore systems, so that access to main memory could be delegated to accelerators (at varying levels of coupling) via capabilities and capabilities could be used to mediate access to accelerators. In this scheme, it would be possible to provide an encryption coprocessor that would store keys internally and perform encryption and decryption operations on blocks of memory delegated to it from userspace.
This is part of a more general problem, which is that most accelerator designs currently follow the model of a device, which requires kernel mediation to use and don't really support virtualisation well. The design you want is a resource that's cheap to access from userspace and easy to multiplex. Once you have that, it's easy to have useful crypto coprocessors (i.e. ones that cause a speedup for everything, rather than a slowdown for anything smaller than a few tens of KBs).
A lot of routers actually run FreeBSD/MIPS. The FreeBSD MIPS port was jointly done by Juniper and Cisco. The CHERI project is jointly funded by two DARPA programs, CRASH and MRC. The MRC-focussed part is looking as software-defined networking and switch fabrics. Part of this includes putting CHERI on the NetFPGA 10G boards (which have 4 10GigE ports, so can do a reasonable amount of packet pushing). For a switch, you typically want some hardware acceleration of lookup tables for routing and so on, but CHERI might make sense for isolating flows from each other. We'll see over the next year...
I appreciate your comments, but would like to add that it appears to me that correctness+security has by now become more important for applied computer science than processing throughput.
Yes, that's one of the motivations for our work. Now seems to be the time when it's possible to persuade CPU vendors that dedicating a little bit more hardware to security is valuable. It isn't quite the dichotomy that you describe though, because you can either implement security features in software or hardware and doing so in hardware can mean that you see a performance improvement for the sorts of use cases that people increasingly care about.
Currently, Ohloh.net is aware of around 11 billion lines of C/C++ code. I'm including C++ because, while it's possible to write memory-safe C++, all of the evil pointer tricks that are permitted in C are also allowed in C++. Rewriting 11 billion lines of code, even in a language that is 100 times more expressive than C, would be a daunting task. Note that Ohloh only tracks open source code in public repositories: there's a lot more that's in private codebases.
For a fully memory-safe language, you need things like bounds checks on array accesses. Some research from IBM about 10 years ago showed that, with JVMs of the time, the cost of bounds checks accounted for the vast majority of the difference in performance between C++ and Java implementations of the same algorithms. They did some nice work on abstractions that allowed most bounds checks to be statically elided, but their approach never made it into the language. The overhead is somewhat hidden by superscalar architectures (the bounds check is done in parallel and the branch is predicted not-taken), but not on low-power in-order chips like the Cortex A7 that you'll find in a lot of modern mobile phones.
Note that, in our implementation, capabilities are orthogonal to paged memory management. CHERI's capabilities are within a virtual address space. The MMU is a really useful abstraction for an OS wanting to isolate processes from each other and implement things like swapping, lazy allocation, and so on. It's a pretty poor abstraction for implementing memory safety and sandboxing within an application. We regard the two as complementary: CHERI provides both a traditional MIPS TLB and capabilities, and the FreeBSD port uses them both.
We hope so too. One of the things on my short-term todo list is define a set of requirements for our extensions to C so that we can have source compatibility across different CPUs implementing similar ISA extensions. Of course, this is (optimistically) assuming that other CPUs are going to provide similar extensions...
There's a little bit of comparison to the Burroughs architecture that was one of the forerunners of the Unisys architecture in the paper. I'm not overfly familiar with the later Unisys MCP, so this may be wrong:
Our approach was explicitly intended to work with language that are not memory safe (i.e. C and friends). If you have a memory-safe language, then there is some cost associated with enforcing the memory safety in software, which CHERI can assist with, but you don't see much of a win.
As soon as you start mixing languages, you get the worst of all worlds. A typical Android application is written in C and Java (and some other things) and so gets all of the memory safety of C, plus all of the performance of Java. A single pointer error in the C code can corrupt the Java heap. One of my students last year implemented a modified JNI on CHERI that allows this sort of mixing but without the disadvantages. Java references are passed to C code as sealed capabilities, so the C code can pass them back to the JVM, but can't dereference them. The C code itself runs in a sandbox (C0 - the capability register that identifies the region of memory that normal MIPS loads and stores can use - is restricted to a smallish subset of the total [virtual] address space) so any pointer errors inside the C are constrained to only touch the C heap (of that particular sandbox - there can be a lot). He demonstrated running buggy and known-exploitable code in the JNI, without it compromising the Java code, with a very small overhead for calling in and out of the sandbox. Most interestingly, he was also able to enforce the Java security model for native code: the sandboxed code couldn't make system calls directly and had to call back into the currently active Java security monitor to determine whether it was allowed to.
Another of my students implemented an accurate (copying, generational) garbage collector for capabilities. This can be used with C code, as long as the allocator is outside of the normal C0-defined address space (otherwise pointers can leak as integers into other variables and be reconstructed). In particular, you can use this to track references to high-level language objects as they flow around C code and either invalidate them or treat them as GC roots, so you don't get dangling pointer errors. Or you can just use his allocator in C and have fully GC'd C code...
My understanding of MCP is that it uses high-level languages in the kernel, but does nothing to protect (for example) typesafe ALGOL code from buggy C code within the same userspace process.
The processor is implemented as a softcore in BlueSpec SystemVerilog, which is a high-level hardware description language (HDL). The source code can be compiled to C for simulation, so you can run it on a general-purpose CPU. We get around 30K instructions per second doing this. It can also be compiled to verilog and then synthesized into gate layouts for an FPGA. We can run at 100MHz (probably 125MHz, but we don't push it) in an Altera Stratix IV FPGA, with around 1 instruction per clock (a bit less), so around 3000 times faster than simulation.
In theory, you could also take the verilog and generate a custom chip. In practice, you wouldn't want to without some tweaking. For example, our TLB design is based on the assumption that TCAMs are very expensive but RAM is very cheap. This is true in an FPGA, but is completely untrue if you were fabbing a custom chip.
Although we use the term 'source code', it's perhaps better to think of it as the code for a program that produces a design of a processor, rather than the source code for a processor.
In terms of software patents, there's some annoying precedent that a software implementation of a architectural patent can be infringing. The MIPS architecture that we implement has LWR and LWL instructions that accelerate unaligned loads and stores. These were patented (the patents have now expired) and the owners of the patent won against someone who created a MIPS implementation where these two instructions caused illegal instruction traps and were emulated in software. The software implementations were found to infringe the hardware patent.
Impressive is also the dare to implement a capability based model with some backing that could lead to consumer products eventually. How many decades this has been put to side now in the general market?
Most of the early capability systems (with the exception of the M-Machine from MIT) were extreme-CISC chips. They implemented complex capability walking so you could have multiple levels of indirection before you got to the . We aim to provide enough hardware support that you can implement these things in software, in a way that can't be bypassed. We're also aiming for a single hardware abstraction that scales from per-object bounds checking up to large sandboxes (think NaCl, but cheap and hardware-enforced).
The limitation of the M-Machine was that it required power-of-two sized objects, so you ended up with a huge amount of padding (very bad for caches, TLBs, and everything else).
In terms of consumer products... wait and see. We're exploring a few exploitation routes, but there's nothing concrete. Ask again in a year or two...
I'm one of the authors of the paper (I did the LLVM support, some of the ISA design and the hardware implementation, and a token amount of OS work), so I'm happy to answer questions about it.
Yes, we agree. That's why we have a fairly simple set of instruction set extensions requiring no complex logic (TCAMs, associative structures, complex data structure walkers, and so on) that we have formally verified provide the required security properties.
To be fair to the submitter, Theo did some amazing work with the laser cutter to produce the tablets. Mine doesn't have the SRI and Cambridge logos etched into the front (or a battery, actually - it's one of the first models) but the it's still very nice. Not really competitive with the iPad, but definitely something we can plausibly use as a prototype.
Tipping still happens in France, but service was required to be included in the bill in the '90s so it's now solely to reward unusually good service, rather than the implicit surcharge that is present on most services in the USA.
The economics of solar power are complicated. The technology is improving sufficiently fast that it doesn't make sense to deploy it in large scale now, because if you wait a year then your ROI will be sooner. Unfortunately, if no one deploys it now, then the newer cells won't make it into production. This is why there are a lot of subsidies floating around for solar power, to make it a good idea for people with some spare capital to deploy now so that in 10 years it will make economic sense for everyone.
$5 seems a lot. I pay Amazon £7.28/month for DVD rentals. This gives me 3 disks at home at a time and unlimited rentals. I can usually return them at the weekend and midweek if I'm feeling lazy and watching a lot, or just once a week if I'm a bit more busy. Going with the second, that's 12 films a month for £7.28, or 60p (a bit under $1) per movie. TV shows typically come 2-4 episodes on a disk, so divide by 2 or 4 for episodes.
I'd happily pay £10-15 per month for a service that had the same range as a DVD rental service, provided DRM-free streams in a standard format, and had a reasonable download cap (maybe 30 hours per month for £10, 60 for £15, something on that order).
I don't want to pay for a movie, I want to pay for access to a large library of movies that I can watch however I want, on any device I want.
I want to give the pizza guy a $2 tip. Do I really want to get his email address, register online, have him register online, get his mobile phone number and all of that.
Well, in the UK, tipping the pizza guy is pretty rare - you're paying for the service already (and delivery drivers are covered by the same minimum wage laws so they get a salary that you can live on). Credit card terminals in restaurants typically provide a tip field so that you can add a tip on at the end.
How is your suggestion even in the same universe as "more convenient than cash"?
Most of the people I'd want to send cash to are people I know and are already in my phone's contact list.
If you live in a culture that is fundamentally opposed to paying people a reasonable wage or stating up-front how much things cost, then the solution would be to have a QR code on the box with a note saying 'Did you get good service? Send a tip here' and the details required to receive the payment. Rather than having to find the $2, you'd just wave the card in front of your phone, select the amount, and hit send. The driver wouldn't need to carry cash.
I'm personally happy to accept the benefits of an inflationary currency (incentivising investment, penalising cash-hoarding). If you're not, then I suggest that you go and live in a country with a non-inflationary currency for a while and see how you like it.
In the UK, Barclays has offered a system called PingIt for a while that lets you send money to anyone with an email address. There's now a system called PayM that most of the banks have opted into that lets you send money to someone using their mobile phone number. The receiver just needs to register their mobile number and account. I'm not convinced by the security yet, but it's more convenient than cash for paying people.
Slashdot Mirror
Apple was definitely behind the power, performance curve," said Nathan Brookwood, principal analyst at Insight 64. The PowerPC processor that emerged from that earlier pairing changed that
PowerPC was pushed by the AIM alliance: Apple, IBM, Motorola. The latter two developed and produced chips. Apple had some input. The goal was an ISA that made it easy to emulate both m68k and i386.
Yup, it's on GitHub in two repositories, one for LLVM and one for clang. We've been pushing the fixes for MIPS IV upstream, but there are some changes to the mid-level optimisers to make them aware that not all pointers are integers and some extensions to the platform-independent code generators for the same.
We've upstreamed all of the FreeBSD changes required for the base processor (BERI) and the development boards, so FreeBSD 10 will run out of the box on CHERI (it just won't expose any of the capability features). The CHERI-enabled version of FreeBSD is also on GitHub.
The code for the processor is just in a tarball at the moment. We're going to move it to some sensible public revision control system once we've worked out how much of our internal svn history wants to be included in the public release.
This is an interesting design. Have you thought about maybe posting seL4 to it once that's open-sourced in a couple of weeks? /akr
Yes, we're looking forward to the seL4 release. Somewhat cautiously, as there's no mention of a license (if it's GPL'd, we probably won't put any effort into it). In CHERI, we have store-capability and load-capability permissions on both capabilities and pages. To implement the seL4 model (where capabilities are stored in special pages), we'd only need to use the TLB-grandularity protections, but it would be more interesting to provide fine-grained capability storage, which would reduce a bit of the difficulty in programming for seL4.
We're also interested in the impact on the proofs in seL4. For example, seL4 currently assumes that the TLB is correct as an axiom (read the errata sheet for any production CPU to see how valid that is). With CHERI, we have some proofs extending down into the ISA and into the implementation and it would be interesting to see how these could be coupled to have a proven-secure systems. The folks at UT Austin and Centaur are (independently of us, but we talk to them periodically) working hard on proving that properties of higher-level HDLs are mapped correctly to gate layouts, so in a few years it may be possible to have a microkernel with security properties proven correct right down to the gate level.
Ilya Bakulin is currently working on porting FreeBSD to run atop L4 and is interested in extending this port to run on seL4. That would also be an interesting approach when combined with CHERI, allowing a full FreeBSD to run with some critical services moved out and into a tiny TCB.
We haven't specifically looked at key storage, but we are looking at potential ways of extending the approach to heterogeneous multicore systems, so that access to main memory could be delegated to accelerators (at varying levels of coupling) via capabilities and capabilities could be used to mediate access to accelerators. In this scheme, it would be possible to provide an encryption coprocessor that would store keys internally and perform encryption and decryption operations on blocks of memory delegated to it from userspace.
This is part of a more general problem, which is that most accelerator designs currently follow the model of a device, which requires kernel mediation to use and don't really support virtualisation well. The design you want is a resource that's cheap to access from userspace and easy to multiplex. Once you have that, it's easy to have useful crypto coprocessors (i.e. ones that cause a speedup for everything, rather than a slowdown for anything smaller than a few tens of KBs).
I hope no one misses the irony of using a DARPA-funded network protocol to post about how evil DARPA is...
A lot of routers actually run FreeBSD/MIPS. The FreeBSD MIPS port was jointly done by Juniper and Cisco. The CHERI project is jointly funded by two DARPA programs, CRASH and MRC. The MRC-focussed part is looking as software-defined networking and switch fabrics. Part of this includes putting CHERI on the NetFPGA 10G boards (which have 4 10GigE ports, so can do a reasonable amount of packet pushing). For a switch, you typically want some hardware acceleration of lookup tables for routing and so on, but CHERI might make sense for isolating flows from each other. We'll see over the next year...
I appreciate your comments, but would like to add that it appears to me that correctness+security has by now become more important for applied computer science than processing throughput.
Yes, that's one of the motivations for our work. Now seems to be the time when it's possible to persuade CPU vendors that dedicating a little bit more hardware to security is valuable. It isn't quite the dichotomy that you describe though, because you can either implement security features in software or hardware and doing so in hardware can mean that you see a performance improvement for the sorts of use cases that people increasingly care about.
Currently, Ohloh.net is aware of around 11 billion lines of C/C++ code. I'm including C++ because, while it's possible to write memory-safe C++, all of the evil pointer tricks that are permitted in C are also allowed in C++. Rewriting 11 billion lines of code, even in a language that is 100 times more expressive than C, would be a daunting task. Note that Ohloh only tracks open source code in public repositories: there's a lot more that's in private codebases.
For a fully memory-safe language, you need things like bounds checks on array accesses. Some research from IBM about 10 years ago showed that, with JVMs of the time, the cost of bounds checks accounted for the vast majority of the difference in performance between C++ and Java implementations of the same algorithms. They did some nice work on abstractions that allowed most bounds checks to be statically elided, but their approach never made it into the language. The overhead is somewhat hidden by superscalar architectures (the bounds check is done in parallel and the branch is predicted not-taken), but not on low-power in-order chips like the Cortex A7 that you'll find in a lot of modern mobile phones.
Note that, in our implementation, capabilities are orthogonal to paged memory management. CHERI's capabilities are within a virtual address space. The MMU is a really useful abstraction for an OS wanting to isolate processes from each other and implement things like swapping, lazy allocation, and so on. It's a pretty poor abstraction for implementing memory safety and sandboxing within an application. We regard the two as complementary: CHERI provides both a traditional MIPS TLB and capabilities, and the FreeBSD port uses them both.
We hope so too. One of the things on my short-term todo list is define a set of requirements for our extensions to C so that we can have source compatibility across different CPUs implementing similar ISA extensions. Of course, this is (optimistically) assuming that other CPUs are going to provide similar extensions...
There's a little bit of comparison to the Burroughs architecture that was one of the forerunners of the Unisys architecture in the paper. I'm not overfly familiar with the later Unisys MCP, so this may be wrong:
Our approach was explicitly intended to work with language that are not memory safe (i.e. C and friends). If you have a memory-safe language, then there is some cost associated with enforcing the memory safety in software, which CHERI can assist with, but you don't see much of a win.
As soon as you start mixing languages, you get the worst of all worlds. A typical Android application is written in C and Java (and some other things) and so gets all of the memory safety of C, plus all of the performance of Java. A single pointer error in the C code can corrupt the Java heap. One of my students last year implemented a modified JNI on CHERI that allows this sort of mixing but without the disadvantages. Java references are passed to C code as sealed capabilities, so the C code can pass them back to the JVM, but can't dereference them. The C code itself runs in a sandbox (C0 - the capability register that identifies the region of memory that normal MIPS loads and stores can use - is restricted to a smallish subset of the total [virtual] address space) so any pointer errors inside the C are constrained to only touch the C heap (of that particular sandbox - there can be a lot). He demonstrated running buggy and known-exploitable code in the JNI, without it compromising the Java code, with a very small overhead for calling in and out of the sandbox. Most interestingly, he was also able to enforce the Java security model for native code: the sandboxed code couldn't make system calls directly and had to call back into the currently active Java security monitor to determine whether it was allowed to.
Another of my students implemented an accurate (copying, generational) garbage collector for capabilities. This can be used with C code, as long as the allocator is outside of the normal C0-defined address space (otherwise pointers can leak as integers into other variables and be reconstructed). In particular, you can use this to track references to high-level language objects as they flow around C code and either invalidate them or treat them as GC roots, so you don't get dangling pointer errors. Or you can just use his allocator in C and have fully GC'd C code...
My understanding of MCP is that it uses high-level languages in the kernel, but does nothing to protect (for example) typesafe ALGOL code from buggy C code within the same userspace process.
In theory, you could also take the verilog and generate a custom chip. In practice, you wouldn't want to without some tweaking. For example, our TLB design is based on the assumption that TCAMs are very expensive but RAM is very cheap. This is true in an FPGA, but is completely untrue if you were fabbing a custom chip.
Although we use the term 'source code', it's perhaps better to think of it as the code for a program that produces a design of a processor, rather than the source code for a processor.
In terms of software patents, there's some annoying precedent that a software implementation of a architectural patent can be infringing. The MIPS architecture that we implement has LWR and LWL instructions that accelerate unaligned loads and stores. These were patented (the patents have now expired) and the owners of the patent won against someone who created a MIPS implementation where these two instructions caused illegal instruction traps and were emulated in software. The software implementations were found to infringe the hardware patent.
Impressive is also the dare to implement a capability based model with some backing that could lead to consumer products eventually. How many decades this has been put to side now in the general market?
Most of the early capability systems (with the exception of the M-Machine from MIT) were extreme-CISC chips. They implemented complex capability walking so you could have multiple levels of indirection before you got to the . We aim to provide enough hardware support that you can implement these things in software, in a way that can't be bypassed. We're also aiming for a single hardware abstraction that scales from per-object bounds checking up to large sandboxes (think NaCl, but cheap and hardware-enforced).
The limitation of the M-Machine was that it required power-of-two sized objects, so you ended up with a huge amount of padding (very bad for caches, TLBs, and everything else).
In terms of consumer products... wait and see. We're exploring a few exploitation routes, but there's nothing concrete. Ask again in a year or two...
I'm one of the authors of the paper (I did the LLVM support, some of the ISA design and the hardware implementation, and a token amount of OS work), so I'm happy to answer questions about it.
Yes, we agree. That's why we have a fairly simple set of instruction set extensions requiring no complex logic (TCAMs, associative structures, complex data structure walkers, and so on) that we have formally verified provide the required security properties.
To be fair to the submitter, Theo did some amazing work with the laser cutter to produce the tablets. Mine doesn't have the SRI and Cambridge logos etched into the front (or a battery, actually - it's one of the first models) but the it's still very nice. Not really competitive with the iPad, but definitely something we can plausibly use as a prototype.
Tipping still happens in France, but service was required to be included in the bill in the '90s so it's now solely to reward unusually good service, rather than the implicit surcharge that is present on most services in the USA.
The economics of solar power are complicated. The technology is improving sufficiently fast that it doesn't make sense to deploy it in large scale now, because if you wait a year then your ROI will be sooner. Unfortunately, if no one deploys it now, then the newer cells won't make it into production. This is why there are a lot of subsidies floating around for solar power, to make it a good idea for people with some spare capital to deploy now so that in 10 years it will make economic sense for everyone.
$5 seems a lot. I pay Amazon £7.28/month for DVD rentals. This gives me 3 disks at home at a time and unlimited rentals. I can usually return them at the weekend and midweek if I'm feeling lazy and watching a lot, or just once a week if I'm a bit more busy. Going with the second, that's 12 films a month for £7.28, or 60p (a bit under $1) per movie. TV shows typically come 2-4 episodes on a disk, so divide by 2 or 4 for episodes.
I'd happily pay £10-15 per month for a service that had the same range as a DVD rental service, provided DRM-free streams in a standard format, and had a reasonable download cap (maybe 30 hours per month for £10, 60 for £15, something on that order).
I don't want to pay for a movie, I want to pay for access to a large library of movies that I can watch however I want, on any device I want.
You didn't read any of the rest of my post, did you?
I want to give the pizza guy a $2 tip. Do I really want to get his email address, register online, have him register online, get his mobile phone number and all of that.
Well, in the UK, tipping the pizza guy is pretty rare - you're paying for the service already (and delivery drivers are covered by the same minimum wage laws so they get a salary that you can live on). Credit card terminals in restaurants typically provide a tip field so that you can add a tip on at the end.
How is your suggestion even in the same universe as "more convenient than cash"?
Most of the people I'd want to send cash to are people I know and are already in my phone's contact list. If you live in a culture that is fundamentally opposed to paying people a reasonable wage or stating up-front how much things cost, then the solution would be to have a QR code on the box with a note saying 'Did you get good service? Send a tip here' and the details required to receive the payment. Rather than having to find the $2, you'd just wave the card in front of your phone, select the amount, and hit send. The driver wouldn't need to carry cash.
I'm personally happy to accept the benefits of an inflationary currency (incentivising investment, penalising cash-hoarding). If you're not, then I suggest that you go and live in a country with a non-inflationary currency for a while and see how you like it.
Paying a monthly fee for a bank account seems somewhat archaic. Let me guess: you live in the USA?
In the UK, Barclays has offered a system called PingIt for a while that lets you send money to anyone with an email address. There's now a system called PayM that most of the banks have opted into that lets you send money to someone using their mobile phone number. The receiver just needs to register their mobile number and account. I'm not convinced by the security yet, but it's more convenient than cash for paying people.
The Jurassic period. O2 in atmosphere was 130% modern levels
And all of the species that were dominant during the Triassic period did really well throughout the Jurassic...