Sensible thought. I very much hope there is a glut of one-three year old ram and GPUs on the market in about one year when the AI market "cools" and the ear-marked components return to the market.
The banks that lent the AI industry the money are already trying to sell their debt.
Unless you meant buying an entire server (instead of laptop/desktop components), it won't work out the way you are describing. Prices may come down, but the components for the datacenters market aren't fungible with the components for laptop or desktop. You might not know what is being "earmarked" in this case?
Yes, the DRAM dies all come from the same wafer supply and fab capacity, and those limits are the cause of the current prices. However, once the memory OEMs have packaged DRAM dies into something like an LRDIMM or SOCAMM, the cake is baked. It's no longer usable in a laptop or desktop. No amount of X-year-old LRDIMMs (hypothetically) flooding the market will be useful for anyone's desktop or laptop. And then there's HBM, where the dies are directly on-package with the CPU or GPU.
Second-hand, revalidated server DRAM components may contribute somewhat to a price decrease, but those won't be the components you or I will be purchasing (unless you run a true server platform as a desktop, in which case, shine on you crazy diamond!).
The same is partly true for GPUs: there are PCIe versions, but most are OAM or SXM modules. You might be able to jury-rig an SXM module into a desktop? Adapter cards exist for at least some SXM versions, and you could figure out the cooling somehow? But it's probably more trouble than it's worth.
I have some amount of confidence that the sellers on AliExpress will figure out how to create cheap hardware that makes use out of all of this when it hits the secondary market.
For standard desktop CPUs, the memory controller doesn't support the signaling required to communicate with an RDIMM. There's no clever AliExpress adapter that will magically give a component within your CPU capabilities that it simply doesn't have.
However, if you have a true workstation, you don't even need some adapter from AliExpress! Xeon 600, Threadripper 7000, Threadripper 9000 all support RDIMMs natively.
If an AI crash left the market with a glut of RDIMMs, we would absolutely see AliExpress sellers removing the individual DRAM packages from used RDIMMs and installing eight out of ten of them onto empty UDIMMs. The overhead costs would be a quick bit of hot air reflow, a 5V PMIC and associated components, and a new module PCB, and the sellers would get to build 10 UDIMMs for every eight RDIMMs they scavenge.
Whenever something like this happens the shortage is exacerbated by hoarders and get rich types. For example used Mac Minis are almost certainly going to find another widespread use or be really underpriced in the secondary market.
Maybe not insurmountable in the "laws of physics" sense?
However, it may be so costly in cobbled-together parts, and in time to deal with unsupported drivers and/or VBIOS, that it is not worth it compared to using a proper server hardware with proper SXM sockets. Nvidia is also doing a lot of new, exciting things with networking that may make their SXM-based GPUs require even more just-so hardware support as time goes on.
Getting older SXM3 GPUs (i.e. for V100's, from 2017) working via PCIe adapters has been done reliably. However, here is someone who did that successfully, and spent a chunk of time last year trying to do the same for SMX5 (H100) and failing:
We were all spoiled by the era from 2005-2020, when you could squint at a "server" configuration and see that it was expensive, high-binned, commodity hardware with some extra RAS and OOBM features. You could buy parts harvested from a retired server based on Xeon E5-1680v3 and drop them in your workstation. Or you could buy an entire single-socket Xeon E5 v3 server, plop in a GPU, and mostly use it as a workstation! As-is!
Now the serious servers have SOCAMM memory, 400Gbps networking, EDSFF SSDs, maybe are configured for CXL 3.0, et cetera. The hardware itself is so divergent that it isn't swap-in-plug-and-play with desktops, even for some high-end workstations.
I really appreciate your comment and the difficulties, but if we're hypothetically talking about $100k hardware for pennies on the dollar.. it looks like the problems are all analog? Like ok, add power supply on top of form factor and cooling but all of the silicon is compatible, right? If (if!) we are talking about pennies on the dollar then those problems are solvable.
For example, if I am a homelab, I don't necessarily need the integrated SFP networking stuff to work, I'm happy with my single overpowered GPU. I don't need CXL either, I just want one badass H200 running in my rig. Maybe Shenzhen will productive that?
There already are SXM5-to-PCIe adapter cards, but from what little I've seen, they are all a bit dodgy. If (if!) there really is some flood of used/decommissioned H100s/H200s into the market, perhaps that will motivate iterations and mass production of these, and they become more reliable?
It is not 100% out of the question. I would also love to see it happen, but am being realistic about the chances, as a check on my own usual optimism.
I think the more likely scenario is, rather than a flood of decommissioned Hopper-era servers being separated into parts, and the parts sold for "pennies on the dollar", we will see entire 4x H100 servers (CPU and RAM included) sold for "dimes on the dollar" to small-and-mid-sized businesses and labs. A formerly-$250K server sold for $50K after six years. Or $20K after ten years. Something like that.
For a while, I programmed while walking on a mini-laptop. Nice walking paths where I lived. I was on a hobby project and wanted to spend any minute on it. It wasn't pretty. I kept trying to design a contraption I could wear on my shoulders that worked like a laptop desk.
I also attached a laptop to a treadmill at home, but the static electricity from the rubber mat kept zapping the laptop.
The best result was a laptop on an exercise bike. But the bike couldn't have a high resistance or I would lose concentration.
I have an under-desk bike (just pedals really). Being able to just move my feet while working is nice. But yeah once it turns into an actual workout then I'd be focusing on pedaling and not work.
Marked me down? That was a joke about how quotes are misattributed on the internet. (I am the following person) But if you are good at your job because you see faults before humour and it happens as a personal trait, then good on you. Found another fault.
> Diversion.—When I have occasionally set myself to consider the different distractions of men, the pains and perils to which they expose themselves at court or in war, whence arise so many quarrels, passions, bold and often bad ventures, etc., I have discovered that all the unhappiness of men arises from one single fact, that they cannot stay quietly in their own chamber. A man who has enough to live on, if he knew how to stay with pleasure at home, would not leave it to go to sea or to besiege a town. A commission in the army would not be bought so dearly, but that it is found insufferable not to budge from the town; and men only seek conversation and entering games, because they cannot remain with pleasure at home.
> But on further consideration, when, after finding the cause of all our ills, I have sought to discover the reason of it, I have found that there is one very real reason, namely, the natural poverty of our feeble and mortal condition, so miserable that nothing can comfort us when we think of it closely.
> Whatever condition we picture to ourselves, if we muster all the good things which it is possible to possess, royalty is the finest position in the world. Yet, when we imagine a king attended with every pleasure he can feel, if he be without diversion, and be left to consider and reflect on what he is, this feeble happiness will not sustain him; he will necessarily fall into forebodings of dangers, of revolutions which may happen, and, finally, of death and inevitable disease; so that if he be without what is called diversion, he is unhappy, and more unhappy than the least of his subjects who plays and diverts himself.
> Hence it comes that play and the society of women, war, and high posts, are so sought after. Not that there is in fact any happiness in them, or that men imagine true bliss to consist in money won at play, or in the hare which they hunt; we would not take these as a gift. We do not seek that easy and peaceful lot which permits us to think of our unhappy condition, nor the dangers of war, nor the labour of office, but the bustle which averts these thoughts of ours, and amuses us.
> Reasons why we like the chase better than the quarry.
> Hence it comes that men so much love noise and stir; hence it comes that the prison is so horrible a punishment; hence it comes that the pleasure of solitude is a thing incomprehensible. And it is in fact the greatest source of happiness in the condition of kings, that men try incessantly to divert them, and to procure for them all kinds of pleasures.
> The king is surrounded by persons whose only thought is to divert the king, and to prevent his thinking of self. For he is unhappy, king though he be, if he think of himself.
> This is all that men have been able to discover to make themselves happy. And those who philosophise on the matter, and who think men unreasonable for spending a whole day in chasing a hare which they would not have bought, scarce know our nature. The hare in itself would not screen us from the sight of death and calamities; but the chase which turns away our attention from these, does screen us.
As a software developer, when I used to smoke, I would go outside for 5-10 minutes and talk shit with other regulars who had nothing to do with my work. It was a break that would fix my fading focus (probably for more than one reason :)
I could only keep peak thinking/designing/developing for about an hour or so. That's peak matrix level with edge cases identified and documented on the way.
I could do OK for a lot longer, but the same quality wasn't there.
The non-smokers would resent us for it, but most of them would go for a half to one hour coffee instead.
As a non-smoker I always took smoke breaks when I worked at an office. It’s the place where you can get the juiciest gossips and insights about the company.
I had to scroll back up to see what this reply was to, to get the full chuckle and yup, I was told frequently by my male parental unit that the top two reasons for having kids was chores and tax deductions. But there's a reason farm families leaned on the large side. The more hands you had helping the less hard things could be while never being easy
I did TDD properly the first time in my Masters Degree (ongoing). It was an eye-opener. Write your program in two different ways to make sure you know the requirements by making their outputs match. That's not me being snarky. It actually works well. Just make sure you can type quickly.
The banks that lent the AI industry the money are already trying to sell their debt.
reply