StorageSearch.com is about thought leadership in the SSD market.
from - what's in the
- how will SSDs change the
Since the 1990s
our readers have accelerated the growth of the SSD market and determined
its essence and onward direction.
|repurposing flash as DRAM,
mystifying the memory choice in DIMMs, extending PCIe fabric and the
strangeness of stealth mode SSD companies which already have revenue|
September 3, 2015 - if, like me, you temporarily stepped aside from the flood
of SSD market related news during the past 4 weeks or so for a vacation or
other reasons then what can I tell you so that you feel more back in control?
(in my mind) the most significant enterprise flash related news in recent years
Memory1 (discussed below).
Much less easy to place within any
particular calendar year and to rate for its significance to future
- however - due to its distinctly vaporware-like aspects and stunning lack
of technical details - was the stage managed unveiling of a newly branded
memory architecture by Intel
and Micron -
which (like the emperor's new
clothes) may or may not become significant for enterprise applications in
2016, or 2017 or 2018 - depending on when we can see it working and depending on
what other competitors are doing at that time.
PCIe SSD watchers -
there were incremental advances in
fabric as PMC-Sierra
showed it could get switches and flash to talk the same language together
across rack boundaries using NVMe and RDMA.
As you'd expect - the
usual sprinkle of SSD companies emerged from stealth mode.
included not just VC
funded revenue hopeful startups but also some significant customer
revenue funded companies too - which had developed quietly in China and
decided that maybe the rest of the world should get to hear about them too.
can read more about these and other recent stuff in
1" SSDs (includes BGA,
DOMs & µSSDs)
1.8" SSDs (includes
11 key SSD design
2012 - changes in SSD
changes in SSD market
changes in SSD market
2015 - what might change?
|About the publisher -
1991 to 2015
in the SSD market
Adaptive R/W &
DSP ECC in SSDs
Analysts - SSD market
Animal brands in
and blogs - top 50
Architecture guide - storage
auto tuning, caching,
- SSD enhanced
management in flash SSDs
Banner ads for SSDs
Big versus Small -
in SSD architecture
from SSD leaders
- citing StorageSearch.com
Strategies in the SSD market
Cache ratios inside
software and SSD appliances
Calling for an
end to SSD vs HDD IOPS
Can you tell me the best
way to SSD Street?
- storage interface
- SSD on a chip & DOMs
CIO - a day in the life
Cloud storage -
with SSD twists
in the enterprise market
Consumer SSDs guide
Consumer SSDs - why are they
chips for SSDs
|What defines the
personality of the SSD?
Cost of SSDs
Dark matter users in
Data integrity in
|When your SSD breaks...
do you call?
hidden segments in the enterprise
markets and SSD
DSP in flash
SSD controller IP
ratings in enterprise SSDs today?
|Education - re SSDs
Efficiency - comparing
SSDs - overview
Endurance - in
|The risk of flash
wear-out in SSDs is
a kind of "forever war"
which is never
|Fast purge / secure erase
SSDs - market timeline
(failures in time) & SSDs
flash SSD vs RAM SSD
Forecasts for SSD market
Hard disk drives
Hard drives in an
- SSD fans disagree fundamentals
History of SSD
of data storage (all)
|Hostage to the
fortunes of SSD
SSD appliances (enterprise)
syndrome - invisible SSD capacity
Inanimate Power, Speed
& Strength Metaphors
|If they survive manufacturing they'll
survive our industrial customers too.
IOPS and flash SSDs
IOPS Comparisons -
SSDs and HDDs
- flash SSD
Legacy vs New Dynasty
|Killerbyte had a "no surrender"
attitude when it came to attacks on her personal data integrity.
MLC - in SSD jargon
MLC in enterprise
era of the SSD market
MRAM, PCM & other
threats to flash in SSDs
- mentions on this site
Oracle articles - Burleson
Petabyte SSD roadmap
protection in SSDs
Power, Speed &
Strength in SSD brands
|PR Agencies & SSDs
Price of SSDs - past,
present and why?
RAID - and SSDs
RAM memory chips
RAM SSDs versus Flash
SSD capacity - server vs SAN
Reliability - in SSD
games and enterprise SSD boxes
|Can it guess what's in my next magic
Silos in the solid
SLC vs eMLC
Software (for SSDs)
to enterprise SSDs
Symmetry impacts in
zero storage? - groan...
Top SSD companies -
7 years tracker
UlltraDIMM SSDs etc
propositions for buying SSDs
VC funds in storage
Videos - about SSDs
Where are we now
with SSD software?
SSDs killed 20K hard drives
(linkin) - StorageSearch.com
how significant for the enterprise data ecosystem?
and a list of potential competitors too by
editor - August 13, 2015 (updated with competitors)
|Diablo has launched
an aggressive assault on the enterprise server RAM market with the launch of
a new DRAM compatible emulation memory module (upto 256GB each) called "Memory1" (faqs) which can
functionally and competitively replace upto 90% of
DDR-4 DRAM - with higher
density, lower cost flash.
Here's some of what Diablo said in its
"The revolutionary technology packs 4x the
capacity of the largest DRAM modules, delivering greater capability on fewer
servers and lowering datacenter costs by up to 70%.
system memory slots that now hold 128 or 384 gigabytes of DRAM memory can house
up to 4TB of Memory1 and process data-intensive applications that were
previously beyond reach. These efficiencies are driven by Diablos breakthrough
in memory design, which replaces expensive DRAM with low cost, high capacity
flash, while delivering the performance an enterprise demands.
is deployed into standard DDR4 DIMM slots and is compatible with standard
motherboards, servers, operating systems and applications. Memory1 is shipping
now to select customers and will be broadly available this fall."
brings the low cost and high capacity of flash to large-scale enterprise and
datacenter customers. It is ideal for environments that require large memory
footprints per server for workloads, such as big data analytics and complex Web
applications. The average Memory1 use case enables a 4 to 1 server reduction,
and one customer use case requires 90% fewer servers. Diablo will initially
focus on delivering Memory1 to cloud and hyperscale datacenters, which stand to
see significant economic benefits because of their scale."
|Editor's comments:- I spoke to Diablo
about Memory1 in July - but I wasn't able to write about it before due to their
launch date which was while I was on vacation. |
Sorry about that.
But as I said to Diablo - when you've got a technology story like
this - likely to have a big impact on
history - a few days or weeks delay in the retelling of the story don't
My gut feel assessment is this.
Memory1 is the most significant development in the enterprise flash
market in the past 3 years (since the widespread impact of
and DSP controller technology) and Diablo's Memory1 is the most
significant development in the enterprise application server market in the past
8 years (since the
most enterprise SSD installed capacity to flash away from RAM and - in
particular - the launch of the first flash based PCIe SSDs by
But let me clarify something important.
Memory1 is not an
Memory1 is a volatile memory which replaces DRAM with a
You'll still need some real DRAM in your system. Diablo
currently recommends a ratio of no more than 10 to 1 (Memory1 to RAM). And there
are OS limitations too. Currently support is limited to Linux although there are
plans to support other major platforms too.
the big question - how
does flash replace RAM?
Diablo didn't want to say much about how
they repurposing flash.
Diablo told me - "We are not disclosing
the flash technology, only that we are using widely available NAND flash.
We employ several technologies to ensure endurance, an example is that we
deliver volatile storage, which is less taxing on the media than persistent
Diablo declined to reveal how much
they use inside the flash array. But they did answer my question about warranty
- 3 years.
Despite the absence of any definitive internal data I
think I can make some guesses about how Diablo repurposes flash to behave more
like DRAM. (And I'm sure many of you can too.)
Here's what I think is
what is Enterprise DRAM?
At a top level
- DRAM - seen from the applications server point of view - has complex and
changing latency characteristics.
This is because traditionally it is
the sum effect of a bunch of different latencies (as experienced by the
software) due to a hierarchy of latencies which include:-
- memory caches on the CPU chip,
- caches external to the CPU chips,
- real DRAM (the kind of modules you see plugged into the motherboard) and
You've known all that stuff since forever. But changes have
- emulated DRAM (the virtual memory which gets swapped in and out of HDDs and
In the past year or so in articles which I have collected in
are you ready to
rethink enterprise DRAM architecture? - it's been clear that
researchers have been evaluating the memory and server experience and have
realized that - depending on the application software - it's possible to get
similar performance effects to traditional DRAM and CPU sets in other ways.
the bottom level of RAM - things are much simpler.
of a DRAM cell you have a transistor which outputs a logic value which is
dependent on the charge in a capacitor connected to its input. Because this
charge can leak away quickly - DRAM requires a refresh cycle - which means that
the charge has to be restored thousands of times a second.
refresh process which generates most of the heat in a memory and which limits
how many memory cells you can place in a chip.
At the heart of a
flash cell you have a transistor which outputs a logic value which is
dependent on the charge trapped in an insulated part of the chip. Like DRAM the
charge leaks away. But unlike DRAM the charge can stay pretty valid for years.
The upside of this is that flash consumes less power than DRAM so
you can pack more storage transistors in the same size chip.
downside of this fantastic ability to hold charge is that the charging process
itself is very brutal and damaging when new data is written - hence the idea
As we've learned from the adaptive R/W DSP flash
pioneers - you can significantly improve endurance and write speeds by using
less damaging charge writes combined with better ECC.
But what if
you don't care about endurance at all?
At the heart of a
Memory1 flash cell? - If you don't care too much about losing data when the
power is unplugged - then maybe you don't need so much charge - and maybe the
write can be very much quicker.
I don't know for sure what schemes
Diablo uses in Memory1 - and it's probably a combination of 3 or 4 new key
things which are needed to make this work - which are over and above what the
company has already proved with its
But what we've got here is a new memory type - which will not
only take business away from traditional DRAM and CPUs (as a special case
extrapolation of the CPU-SSD
equivalency reasons which I told you about in 2003) but Memory1 will also
take business away from that 3D PCM memory which
Micron have been
about too - for the simple reason that flash is a cheap and more mature
This new technology from Diablo is more significant than
what we had been led to expect from their previews a year ago.
doesn't replace memory
channel SSDs or hybrid
DIMMs - it's a new product type - flash as RAM - which could lead to a
market as significant in revenue as the whole
PCIe SSD market.
guessing then that Diablo will stay the #1 most
researched company in
the SSD ecosystem for a while as a result.
As I said above - many other SSD companies have
been eyeing exactly the same applications problems which Diablo's Memory1 is
designed to satisfy:- lowering the cost of server populations which rely on
access to extreme amounts of low latency RAM in order to fulfill their business
While there are currently no exact form factor matches to
Memory1 which give similar memory density - there are other ways to get similar
results. And if you regard the server box as the socket footprint (which it
really is - if you're planning to buy 10K or 100K servers) then DIMM socket
compatibility - while convenient - is not essential.
list of potential hot competitors in the Diablo Memory1 market space goes like
Flash based solutions.
In the current state of
the memory market - nand flash is the cost leader. Alternative approaches
- software which remaps big RAM memory into flash SSDs.
operates with a wide range of SSD interfaces - so users have a lot of freedom to
optimize their latency / cost profile. At the extreme limits Memory1 may be
faster in latency terms than most PCIe SSDs - but my guess is that some kind of
future ioDrive like product (possibly optimized to strip away non volatility)
would give very similar performance.
SSD software companies
which until recently have been in stealth mode are also coming into the latency
lowering flash market.
processing - combined with PCIe SSD platform.
Companies which have
been talking about this since last year include:-
And earlier this year further validation of the concept came from
at MIT who showed that flash can come close to RAM rich server performance
by using clunky app-specific FPGAs combined with flash.
non flash / nvm solutions
- Netlist? - This
company which was embroiled for over a year in legal actions aimed at Diablo's
first product - the
MCS SSDs -
is one of several in the
hybrid DIMM market
which might make a future pitch in this market. Having said that there's a world
of difference between these DIMM compatible RAM SSDs and Memory1 - where the
RAM which is used is actually in another socket supplied by someone else. Other
companies in this category include
Micron. But due to the
market risks of shooting its other products in the foot I think Micron will
prefer to position its response to this application in different ways first
(with higher priced solutions).
These will probably cost more than the flash based competitors.
fabric / flash / nvm alternatives
- Already mentioned above - high density 3D PCM from
Micron. Details at the
present time are sketchy. But it's reasonable to assume this will initially be
offered at much higher cost than flash solutions - because it's a new
technology and unlike Memory1 - a key characteristic it offers is non
contender which converges with all the technologies above is
Express over RDMA includes elements of PCIe fabric but interoperate the
company's low density RAM SSD and flash SSD products.
these can be combined to build low cost low latency fabrics for small server
clusters at a starting price point probably below PLX and much lower (scale)
these fabric solutions enable users to extend the DRAM space at low
microsecond latencies outside the rack rather enabling more RAM in the same
rack. But I think they could also be viable technology launchpads for future
PCIe connected flash memory similar to Diablo's Memory1.
the enterprise SSD story
why's the plot so complicated?by
editor - June 17, 2015
and was there ever a best time to
|Can I tell you any
single most useful thing extracted from all the thousands of
incidental things and trivia I've learned from the
answering readers' questions about the market and in constructing major articles
- I've sometimes asked myself - how do I know this stuff?
explanation which I gave in an earlier
article is that if
you spend decades of time thinking a lot about a single subject and while doing
that talk to most of the experts in that subject - and also talk to a lot of
other people who have their own reasons to be interested - then something does
rub off and stick.
for understanding the SSD market - I don't recommend it to anyone else.
not scalable and not amenable to replication. And there are faster and
more efficient ways for you to learn most of what you need to know. Which I
hope includes trawling sites like this one.
So what have I learned? And
what's the single most useful thing?
From a technology point of view -
the technology is still
And how SSDs interact with other parts of your data
processing assets still
has many adaptations and evolutions to get through before things can
The reason the
continues to be so complicated is this.
In the 1970s when large
scale integrated silicon technology began to disrupt the computer market with
devices like the microprocessor and DRAM - they gave birth to a new generation
of software companies. When using the earliest microprocessors you had to
write all your own software. But whatever you did with the newest CPUs and
memories was invariably cheaper than what had been done before. By the mid to
late 1980s - these systems were faster too - and were starting to replace
enterprise servers - and to rely on a complex ecosystem of software bigger than
anything which had been seen before.
I was lucky in my observation
point in those days to have explored the benefits of applying permutations of
multiple processors, embryonic RAID and solid state storage as performance
accelerators in real-time environments which included 3G databases and Unix
It was a decade later - when reporting on the benefits of
enterprise SSDs here on this site in the late 1990s and early 2000s - that I
realized that somehow the insights about SSDs as applications accelerators -
were not widely appreciated.
The next few paragraphs provides a
summary of that early accidental eureka moment for me - from my
start of some bio stuff - related to this article
SSDserver equivalence idea
didn't seem like a big deal to me at the time (about 1988) as storage was
just one of many bottlenecks
my customers needed to solve.
When I was recruited into a startup
called Databasix in the late 1980s - the founders could already see that new
opportunities were being created to do new things based on AI and database
technology - but that doing useful things in real time would only be possible by
massive parallelization of CPUs (which was not a commodity at that time) and
parallelization of disks (RAID for throughput) and faster storage for the
latency critical parts (solid state storage).
So I and my team spent
all our time playing with the fastest technology toys we could borrow (evaluate)
or buy or modify and figuring out what the nature of bottlenecks were and how
best to solve them in a virtualized way if possible - because many of these
toys were short lived and almost every customer project changed a lot from the
time it was conceived to the time it was delivered. And the only way to make it
work as a business would be to make most of the core software reusable.
- then as now - there were only a small number of users who needed such crazy
Finding the performance fanatics with budgets was
the "business model." Our sales people found some. Which was amazing
in itself as that was before the days of internet marketing. We learned a lot
in a short space of years by providing platforms to researchers in the
industrial, military, seismic research and related markets and also an
innovative and long lived production broadcast program routing system for a
major broadcaster too. It was a lot of work. We wrote all our own drivers for
everything. And our objective was always to reach wire speed of whatever device
end of the bio stuff - (I'm not looking for
another job BTW)
A big new hypothetical question for SSD market thinkers?
of which - above- is the long way of introducing a question which
has occurred to me often recently.
Was there ever an ideal earlier
time and opportunity to inject SSD DNA into enterprise computing architecture
(and OSes) which could have prevented the buildup of complex integration and
associated rip and replace and bypass surgery - of so many layers of
systems software - which have inevitably followed?
I'm not sure.
Very few people realized the value of doing this even in 1988. And in
those days if you tried to integrate SSD acceleration yourself - it involved a
lot of analysis and rewriting OS kernels to make it work in a useful way.
the software industry didn't do it.
And before then there wouldn't
have been a clear need.
And after 1988 (and throughout the subsequent
decade) as the enterprise became fascinated and then bewitched by
most CPU designers were hell bent on a course to just add more cores and DRAM
and wider memory busses and faster clocks as the simplest way to keep getting
So looking back now - it's clear that the standard models
for computer hardware and software architecture in the silicon chip age had
evolved for over 40 years without the concept of another economically
useful latency layer between hard drives and DRAM.
That didn't stop
SSD pioneers from trying to break in. But it was through cracks and rare
business opportunities in which nearly every SSD sale required a huge customer
It was as late as
2009 that SSD
accelerators became offered as a standard option by all mainstream enterprise
server vendors and it was from that time onwards that we saw the
birth of a
true SSD software
You can see why the enterprise SSD market is the most
bewildering part of the enterprise market.
It's not about flash
SSDs are changing a market (data processing) which was
designed without any original conception of SSDs being there in the first
modern era -
SSDs began sneaking into purchase orders whenever the gaps (what was possible
with SSDs compared to what was possible without) looked easy enough and
lucrative enough to edge into.
And the future of the story (as I've
said many times before) is that future data architectures will be managed by
reference to their different interconnected storage assets (SSD) rather than
(as in the past) by reference to their servers.
And all the critical
costs and management decisons will be about how to control the cost of the SSDs.
Note I don't use the term "flash" here deliberately -
because the enterprise SSD story started before flash was an enterprise
technology and will most likely persist till after too.
somewhere at the beginning of this blog I hinted that I might say something
useful about all the stuff I've learned from talking to so many founders of SSD
companies and visionnaries in the market.
Surprisingly the one
enduring and universally useful true thing I've learned isn't about the
(Today's hot chip management technology looks like
steam punk when viewed from the future.)
No it's this...
talking to long time SSD pioneers and serial company founders and those whose
creations have transformed this industry of ours - I often ask - why are you
still doing this? It can't be the money. Surely you could just retire and go
fishing or start a boutique investment company.
The answer every time
is something like this.
The SSD market is the most exciting place to be
working right now. Why would you want to be somewhere else?
For me too
- it's the challenge of figuring out where all these roads are going and what
will the landscape look when the whole territory has been discovered, staked
out, built up, rustled and burned down and then rebuilt again. The unknown and
exciting combined with tantalizingly predictable narratives with often
surprising characters is the SSD story which keeps me going.
conclusion there never was a best time in the past for the enterprise SSD
No past "Heroic Golden Age of SSD".
best time is still now.
|Hmm... it looks like you're seriously
interested in SSDs. So please bookmark this page and come back again soon. |
written thousands of stories and guides related to the SSD market. Some of the
most popular can be
StorageSearch.com is published by
ACSL. © 1992
to 2015 all rights reserved.
Editor's note:- I currently talk to
more than 600 makers of SSDs and another 100 or so companies which are
closely enmeshed around the SSD ecosphere.
Most of these SSD
companies (but by no means all) are profiled here on the mouse site.
I learn about new SSD companies every day, including many in stealth mode. If
you're interested in the growing
big picture of
the SSD market canvass - StorageSearch will help you along the way.
Many SSD company CEOs read our site too - and say they value our thought leading
SSD content - even when we say something that's not always comfortable to hear.
I hope you'll find it it useful too.
never compile email lists from this web site, not for our own use nor anyone
else's, and we never ask you to log-in to read any of our own content on this
web site. We don't do pop-ups or pop-unders nor blocker ads and we
don't place cookies in your computer. We've been publishing on the web since
1996 and these have always been the principles we adhere to.
pages which have Google site search boxes do insert cookies from Google
(not from us).
|SAS SSD market size|
news (September 3, 2015)|
|As with any new memory
technology it will take time and experience to prove whether Optane memory has
enterprise grade reliability.|
| A couple of years ago -
if you were a big company wanting to get into the SSD market by an acquisition
or strategic investment then a budget somewhere between $500 million and $1
billion would have seemed like plenty. |
|VCs & SSDs|
|When you read something
about the SSD market which seems to contradict something which you previously
held as a working hypothesis - what do you do next?
trust SSD market data?|
|In-situ SSD processing -
is about closing important gaps in the intelligence of message passing and the
speed of data access between application processors and SSD controllers.
|12 key SSD
ideas which changed in 2014|
|The SSD market during its
short history (spanning only 40 years) has managed to accrue an imaginative
body of literature which includes truths, half truths, mysticism,
misunderstandings. myths, legends - and in some cases - downright balderdash -
when it comes to the subject of SSD costs, pricing and justifications.
Astrological Age of Enterprise SSD Pricing|
| If you've seen or read -
The Hobbit - then you'll be familiar with the concept of the riddle game.
Something similar is playing out now in the enterprise flash array market. |
|playing the enterprise
SSD box riddle game|
|Even in its infancy -
endurance management was a complicated technical subject - but if we look back
from the perspective from the ultra-complexity of today - it was much easier to
manage and understand. |
|SSD endurance - the
forever war - now in 3D|
|You don't have to
understand the internal details of how these individual techniques work. And
with hundreds of patents already pending in this topic there's a high
probability that the SSD vendor won't give you the details anyway (not even
under NDA). It's enough to get the general idea.|
care management & DSP ECC IP in SSDs|
design of memory controllers, and in particular memory scheduling algorithms,
leads to uncontrolled interference of applications in the memory system"|
|Are you ready to
|"A critical test of
whether you really understand the dynamics of a complex market like enterprise
SSDs - is whether you can predict what rational buyers might do when offered new
product options at the extreme limits of - for example - price."|
Analysis in SSD Market Forecasting|
|Like cosmological dark
matter - which accounts for most of the mass in the universe but which is
invisible to optical telescopes - the SSD dark matter customers will be bigger
in mass than the traditional known enterprise users.
|The big market impact of
SSD dark matter|
|If the flash memory in
your supplier's SSD or array costs 2x as much as that of a competitor - because
it uses more of the same generation chips or uses more expensive
preconditioned older style chips - to do the same job - their profit margins
and your electricity costs will suffer.|
| Efficiency as
|"The winners in SSD
software could be as important for data infrastructure as Microsoft was for
PCs, or Oracle was for databases, or Google was for search."|
|all enterprise data
will touch an SSD |