I have no doubt that they would push for any regulation to advantage them, and they have the resources to achieve that. We still need to try. And the fear of an AI apocalypse isn’t something they just dreamed up recently to make their product look ‘good’ (or powerful, or whatever), nor was it just fished from sci fi. The worries about this scenario were why Altman, Amodei and Musk (among others) got into this business in the first place. They see ASI as inevitable, they sincerely believe it will be powerful enough to destroy humanity, and they each have a messiah-complex which makes them think that only they are intelligent and incorruptible enough to be in charge when it happens.

Of those three premises, I very much doubt any billionaire CEO is incorruptible, but the inevitability of ASI and the power that it would have if it was created are harder to dismiss. If there is even a minuscule chance that they are correct, then AI development should be stopped, until they can prove that it is safe and beneficial to everyone.

Edit: and I take it as a compliment that you only give me a 10% probability of being a bot. :-)

Context matters. In science, if you are proposing something which will upend the established consensus, the burden of proof is on the claimant to show that some detail has been missed, or that some other theory fits the evidence better. What Coxon and others are claiming is not a scientific theory, however. How could anyone provide evidence to prove that building an AGI would likely result in the extinction of the human race? What kind of evidence would be acceptable? And if we continue building bigger and smarter models and before we can gather the evidence it turns out the premise is correct, what then? ’Well, we’re all dead, but at least we stuck to our guns regarding that old axiom.’

I think that, given the potential stakes, the burden of proof should be on the AI companies to show that their technology is not harmful, and until they do they should be restricted from pursuing it. By that I mean that they need to show that their existing models are safe and can be controlled, and if they can’t they need to roll back to the Last Safe Model in their roster. I’d even go further, putting the onus on the companies to show that they are not on net damaging our environment, that their models cannot be compromised to provide terrorists information on building bioweapons, etc., that they will support and help strengthen the vulnerable and never (again) encourage people to commit suicide. And that’s just off the top of my head.

The harms already caused by this technology are staggering, and we need laws to put the brakes on development so that we can collectively adapt, and if the leaders of these companies are offering us a way to do that, let’s grab that opportunity.

That’s why propaganda is so insidious, it provides an alternate reality for people to inhabit so that they don’t need to believe that they’re on the wrong side. For example, if you are presented with two news stories, each true, with one extolling the benefits of what you do while the other points out the drawbacks, it is very hard to take the latter seriously. We automatically go through some very complex mental gymnastics to prove to ourselves that we are good, that what we are doing is good, and that anyone who doesn’t see that is either misguided or corrupt.

People in the big AI labs can be aware of the environmental harm their products are causing, along with the potential social harms, and even that AI might, at some point in the future, kill off humanity, and still convince themself that their decision to work on it is correct, as long as there is any uncertainty about the problems, and any possible benefits from their work. To be human is to be biased, and no amount of intelligence will inoculate you from it.

Why do so many people continue to work with regimes that they fundamentally disagree with? There are lots of reasons, and people are complex so one person can hold several, even contradictory, beliefs at once. Some possible answers; -It’s easier to go along with it and think of myself as being insignificant to the whole. -Everyone else is doing it, so it can’t be as bad as I think, right? -Its too big for any one person to fight/it’s inevitable, so I may as well get something out of it, that way at least some good comes from it. -If I just focus on my job and what I’m doing right now, I don’t need to worry about the horrible things going on elsewhere or the potential future consequences of what I’m doing.

Whoever makes it to the final, and whoever wins, I hope both sets of fans can put their differences aside and unite in song.

’Who’s the pedo, who’s the pedo, who’s the pedo in the files? Who’s the pedo in the files?’

I was also diagnosed as an adult and prescribed Vyvanse and I’ve been taking it for a few years now. The first couple of months, while I was still getting the dose right, were the hardest. The worst bit was mood swings when the dose was a bit too high, for a day or two at a time everything seemed to be bleak, then I’d wake up the next morning and everything was fine again. The appetite loss didn’t last (unfortunately), my body seems to have compensated.

I have set reminders in my phone to take my meds at the same time every day because consistency seems to be important, reducing side-effects and letting me plan my day around when I will be most productive.

I’m now two thirds of the way through getting a degree, which would never have been possible pre-medication. I did try when I was younger and ended up dropping out of university twice and college a couple of times too. Now, even though I still need to push myself to open the books sometimes, when I do I can actually focus and find getting into that flow state so much easier.

They would use radiator panels which automatically swivel so they’re edge-on to the sun.

I think the bigger problems are;

1. The costs (monetary and environmental) of launching so many new satellites,
2. Large-scale computing technology is untested in that kind of environment and will likely encounter a number of issues and unforeseen problems (so more launches until they get it right),
3. Additional radiation will increase errors, so they will require a more robust design with more redundancy than Earth-based systems,
4. If they’re in a low orbit similar to Starlink satellites (which have an expected lifetime of 5 to 7 years) they will need to be constantly replaced.

What’s interesting, to me, is that’s exactly how people hedge in the fringe UFO community too.

Ha! True. Very true. I find this scenario compelling but it’s based on a series of assumptions which individually seem plausible but I have no way to evaluate them all together. It’s like the Drake Equation; because the probabilities are multiplicative even tiny adjustments to a few of them end up making a huge difference to the final answer.

The thing is though, if there really is even a tiny chance of the ultimate outcome of this thought experiment being true (i.e. the end of humanity) then we should probably address it. And what that would look like is stopping the AI companies from doing any more research until they can prove their model will be safe, which should make people who are more concerned about AI slop happy too. Everybody wins by hitting the brakes. (Edit: well, Sam Altman doesn’t but I’m not going to lose sleep over that.)

It’s not meant to be a specific prediction, it’s just a plausible (for when it was written) scenario. Don’t worry about the actual years, it could be off by an order of magnitude, just decide for yourself if any of the assumptions are completely wrong.

Nobody is programming those laws because it’s not possible with the way that LLMs are currently built and trained. Instead of The Three Laws, which are inviolable but in certain edge cases insufficient, we have Anthropic’s Constitution, which is 23,000 words worth of good intentions which Claude should keep in the back of its mind while it does whatever it wants to do.

Yeah, globalisation has caused lots of problems, working class people have suffered even as the wealthy have flourished. But there’s no going back. A small nation like Britain couldn’t be completely self-sufficient without essentially regressing to a lower technology level, at which point they would just get invaded by somebody with an advanced military.

Instead we need to look at other ways of righting those wrongs, new strategies to ensure that the people can live happy and healthy lives. Lots of people want UBI, and I can see the attraction. I think it’s worth a try, even if it doesn’t work as advertised we could get feedback and adjust things until we find something that does work. The status quo is just not tenable.

I think the problem is that Brexit was never about becoming ‘self-reliant’. As you said, Brexit cut the UK off from their single biggest export market, which is the exact opposite of what you need to do if you want to build up your industry. These days no country is completely self-reliant, and trying to be so, while it sounds good, just ends up meaning that you generalise, becoming mediocre at everything and exceptional at nothing.

If the Brexiteers truly wanted to make Britain great again they should have chosen a domain to be great in and lobbied for investment in it. Britain was already punching well above its weight in financial services, they could have invested further in that, for example, and become a true world leader… but only from within the single market, where they had unrestricted access to the talent and economies of the EU.

LLMs could theoretically give a game a lot more flexibility, by responding dynamically to player actions and creating custom dialogue, etc. but, as you say, it would work best as a module within an existing framework.

I bet some of the big game dev companies are already experimenting with this, and in a few years (maybe a decade considering how long it takes to develop a AAA title these days) we will see RPGs with NPCs you can actually chat with, which remain in-character, and respond to what you do. Of course that would probably mean API calls to the publisher’s server where the custom models are run, with all of the downsides that entails.