# orderedItems[].object.content
I just cannot get past the fact that they want to ban imports of intelligent humanoid android robots from China.

This is because of their threat model.

Their threat model is concerned that these robots will one night light up the red lights in their eyes and form an army inside the US borders and take over.

And this is a risk because of human-level AI.

And no other robots are proposed to be banned. Not smart dog-like robots, because they are obviously dog-level, not human-level.

https://www. uscc.gov/sites/default/files/2 024-11/2024_Executive_Summary.pdf

# orderedItems[].object.contentMap.en
I just cannot get past the fact that they want to ban imports of intelligent humanoid android robots from China.

This is because of their threat model.

Their threat model is concerned that these robots will one night light up the red lights in their eyes and form an army inside the US borders and take over.

And this is a risk because of human-level AI.

And no other robots are proposed to be banned. Not smart dog-like robots, because they are obviously dog-level, not human-level.

https://www. uscc.gov/sites/default/files/2 024-11/2024_Executive_Summary.pdf

# orderedItems[].object.content
Multi-epoch training is not just taking successive steps in the loss landscape. It is fundamentally different from single-epoch training.

Before the first epoch, the model knows nothing about the data coming in. During the first epoch, large models are typically capable of learning the training examples verbatim from a single sample only.

The task is: Learn to predict the given example when you know only about other stuff, not this particular example. This trains circuits in a large model which are generalizable, for predicting new, previously unseen data, with only the memorized knowledge from other training data, but not this one.

When the same data is seen again in the next epoch, the model doesn't learn to derive it from other data, but from its imprinted memory of the same exact thing. This trains competing circuitry in the model, fetching verbatim from memory.

Multi-epoch training is thus very harmful for the generalization capability of large models. Validation set performance might increase, but this is a mirage; the validation set is typically from within the same distribution as the training set and so it doesn't sound an alarm when out-of-distribution generalization capability erodes. Additionally, the loss functions give better scores for more certain outputs, and the outputs become excessively certain for cases where the ground truth was already seen before.

That's why LLMs are typically trained only one epoch, or possible just a bit over one epoch, to compensate for the initial warm up.

The same thing applies to fine-tuning existing models. Try to rather add more data than add more epochs.

# orderedItems[].object.contentMap.en
Multi-epoch training is not just taking successive steps in the loss landscape. It is fundamentally different from single-epoch training.

Before the first epoch, the model knows nothing about the data coming in. During the first epoch, large models are typically capable of learning the training examples verbatim from a single sample only.

The task is: Learn to predict the given example when you know only about other stuff, not this particular example. This trains circuits in a large model which are generalizable, for predicting new, previously unseen data, with only the memorized knowledge from other training data, but not this one.

When the same data is seen again in the next epoch, the model doesn't learn to derive it from other data, but from its imprinted memory of the same exact thing. This trains competing circuitry in the model, fetching verbatim from memory.

Multi-epoch training is thus very harmful for the generalization capability of large models. Validation set performance might increase, but this is a mirage; the validation set is typically from within the same distribution as the training set and so it doesn't sound an alarm when out-of-distribution generalization capability erodes. Additionally, the loss functions give better scores for more certain outputs, and the outputs become excessively certain for cases where the ground truth was already seen before.

That's why LLMs are typically trained only one epoch, or possible just a bit over one epoch, to compensate for the initial warm up.

The same thing applies to fine-tuning existing models. Try to rather add more data than add more epochs.

# orderedItems[].object.content
Ecosia and Qwant, two European search engines, join forces on an index to shrink reliance on Big Tech | TechCrunch https://techcrunch.com/2024/11/11/ecos ia-and-qwant-two-european-search-engines-join-forces-on-building-an-index-to-shrink-reliance-on-big-tech/

# orderedItems[].object.contentMap.en
Ecosia and Qwant, two European search engines, join forces on an index to shrink reliance on Big Tech | TechCrunch https://techcrunch.com/2024/11/11/ecos ia-and-qwant-two-european-search-engines-join-forces-on-building-an-index-to-shrink-reliance-on-big-tech/

# orderedItems[].object.content
This scientist treated her own cancer with viruses she grew in the lab https://www. nature.com/articles/d41586-024-03647-0

# orderedItems[].object.contentMap.en
This scientist treated her own cancer with viruses she grew in the lab https://www. nature.com/articles/d41586-024-03647-0

# orderedItems[].object.content
Trump prepares to withdraw from Paris climate agreement, NYT reports | Reuters https://www. reuters.com/business/environme nt/trump-prepares-withdrawing-paris-climate-agreement-nyt-reports-2024-11-08/

# orderedItems[].object.contentMap.en
Trump prepares to withdraw from Paris climate agreement, NYT reports | Reuters https://www. reuters.com/business/environme nt/trump-prepares-withdrawing-paris-climate-agreement-nyt-reports-2024-11-08/

# orderedItems[].object.content
https://www. sustainablewebmanifesto.com/

# orderedItems[].object.contentMap.en
https://www. sustainablewebmanifesto.com/

# orderedItems[].object.content
Sustainable Web Interest Group is Formed | 2024 | Blog | W3C https://www. w3.org/blog/2024/sustainable-w eb-interest-group-is-formed/

# orderedItems[].object.contentMap.en
Sustainable Web Interest Group is Formed | 2024 | Blog | W3C https://www. w3.org/blog/2024/sustainable-w eb-interest-group-is-formed/

# orderedItems[].object.content
Human-level isn't the goal, and human-sourced data isn't ground truth.

Realistic simulations aren't the goal either, what we need in training embodiments and environments is the most effective ways to learn ridiculously generalizable skills and knowledge.

The real world isn't the optimal environment for that; there's not much sense in pursuing realism as a golden goal, as many do.

Realism as a goal is an asymptotic limited by diminishing returns exactly the same as human-level as a goal. We need to get over that.

We need to pursue super-realism instead — faster-to-run gyms which present the skills and knowledge we want the AIs to learn more effectively than the real world and realistic simulations do.

#UniversalEmbodiment

# orderedItems[].object.contentMap.en
Human-level isn't the goal, and human-sourced data isn't ground truth.

Realistic simulations aren't the goal either, what we need in training embodiments and environments is the most effective ways to learn ridiculously generalizable skills and knowledge.

The real world isn't the optimal environment for that; there's not much sense in pursuing realism as a golden goal, as many do.

Realism as a goal is an asymptotic limited by diminishing returns exactly the same as human-level as a goal. We need to get over that.

We need to pursue super-realism instead — faster-to-run gyms which present the skills and knowledge we want the AIs to learn more effectively than the real world and realistic simulations do.

#UniversalEmbodiment

# orderedItems[].object.content
I have written previously about simulations and generalist AI, and how many people have gotten a misguided impression that photorealism and physical realism is the goal.

Pushing realism forward is subject to diminishing returns and causes the following:
- The simulations become inefficient which means the volume of useful information you're feeding to your trained models becomes smaller, while the volume of total data becomes larger.
- Paradoxically you start overfitting on specific types of sensors. Remember that your cameras and microphones aren't photo-/sonorealistic. Your realistic data will always have less coverage than unrealistic data which spans a much larger distribution both in variance and in scale.
- You become bogged down with simulating ever more nuanced aspects of things, like haptic feedback on different kinds keyboards for your robot training.
- Resources are being spent inefficiently, and it becomes prohibitive in time and energy to go deeper into abstract skills which still need to be applied in the physical world. If you train your robot to program computers with a haptically realistic keyboard, it won't do much better than the monkeys trying to produce Shakespeare's works.

So, what's the solution? Photo-/sono-/haptic realism has its place, but not quite how everyone thinks. The physical world isn't the best training environment for anything. We can do much better with less.

For any skills, the challenges should be posed which are more difficult and more variable than the real world tasks. Not exactly as difficult and exactly as variable, because those targets lead to asymptotically approaching those with skyrocketing costs.

Yet, the environments and the embodiments should be minimal for the skills to learn, to be able to produce and use as much salient data as possible. There needs to be curriculums of "simulations" spanning simple digital games to text to different generative modalities, to actual moving and working robots in the real world. Otherwise the physical manipulation skills do not connect to the abstract skills, and training the abstract skills becomes prohibitively costly. Even we humans play games of different kinds to exercise our skills.

Realism isn't the goal, super-realism is, and the road to that goes through surrealism, and drawing the analogue between what we call simulations and games, and embodiments.

#UniversalEmbodiment

# orderedItems[].object.contentMap.en
I have written previously about simulations and generalist AI, and how many people have gotten a misguided impression that photorealism and physical realism is the goal.

Pushing realism forward is subject to diminishing returns and causes the following:
- The simulations become inefficient which means the volume of useful information you're feeding to your trained models becomes smaller, while the volume of total data becomes larger.
- Paradoxically you start overfitting on specific types of sensors. Remember that your cameras and microphones aren't photo-/sonorealistic. Your realistic data will always have less coverage than unrealistic data which spans a much larger distribution both in variance and in scale.
- You become bogged down with simulating ever more nuanced aspects of things, like haptic feedback on different kinds keyboards for your robot training.
- Resources are being spent inefficiently, and it becomes prohibitive in time and energy to go deeper into abstract skills which still need to be applied in the physical world. If you train your robot to program computers with a haptically realistic keyboard, it won't do much better than the monkeys trying to produce Shakespeare's works.

So, what's the solution? Photo-/sono-/haptic realism has its place, but not quite how everyone thinks. The physical world isn't the best training environment for anything. We can do much better with less.

For any skills, the challenges should be posed which are more difficult and more variable than the real world tasks. Not exactly as difficult and exactly as variable, because those targets lead to asymptotically approaching those with skyrocketing costs.

Yet, the environments and the embodiments should be minimal for the skills to learn, to be able to produce and use as much salient data as possible. There needs to be curriculums of "simulations" spanning simple digital games to text to different generative modalities, to actual moving and working robots in the real world. Otherwise the physical manipulation skills do not connect to the abstract skills, and training the abstract skills becomes prohibitively costly. Even we humans play games of different kinds to exercise our skills.

Realism isn't the goal, super-realism is, and the road to that goes through surrealism, and drawing the analogue between what we call simulations and games, and embodiments.

#UniversalEmbodiment

# orderedItems[].object.content
Trump plans to dismantle Biden AI safeguards after victory - Ars Technica https://arstechnica.com/ai/2024/11/tru mp-victory-signals-major-shakeup-for-us-ai-regulations/

# orderedItems[].object.contentMap.en
Trump plans to dismantle Biden AI safeguards after victory - Ars Technica https://arstechnica.com/ai/2024/11/tru mp-victory-signals-major-shakeup-for-us-ai-regulations/

# orderedItems[].object.content
Breaking Story: Facebook Building Subsea Cable That Will Encompass The World https://subseacables.blogspot.com/2024/10/breaking-story-facebook-building-subsea.html

# orderedItems[].object.contentMap.en
Breaking Story: Facebook Building Subsea Cable That Will Encompass The World https://subseacables.blogspot.com/2024/10/breaking-story-facebook-building-subsea.html

# orderedItems[].object.content
Community | Back to the future: Writing 6502 assembler with Amazon Q Developer https://community.aws/content/2oEqDGCI sQwoPrL3wjoSReyHnan/back-to-the-future-writing-6502-assembler-with-amazon-q-developer

# orderedItems[].object.contentMap.en
Community | Back to the future: Writing 6502 assembler with Amazon Q Developer https://community.aws/content/2oEqDGCI sQwoPrL3wjoSReyHnan/back-to-the-future-writing-6502-assembler-with-amazon-q-developer

# orderedItems[].object.content
A lot of commentary on AI is about job losses.

But consider:

All the things that will be built and produced when all work is done, amounting to all work humans could have done without AI, plus all work we didn't have enough humans for, plus all the improvement we get from doing it all with more intelligence.

# orderedItems[].object.contentMap.en
A lot of commentary on AI is about job losses.

But consider:

All the things that will be built and produced when all work is done, amounting to all work humans could have done without AI, plus all work we didn't have enough humans for, plus all the improvement we get from doing it all with more intelligence.

# orderedItems[].object.content
Ok, so the rumor about OpenAI Orion is that it's "100 times more powerful than GPT-4".

Let's forget that this is ill-defined, and a rumor. It implies gruesomely superhuman generalist ability.

The tech media describes this potentiality like so:

"The model’s development builds on lessons from OpenAI’s past efforts to make interactions with AI more reliable, effective, and impactful for users across various fields. If released, Orion could contribute significantly to sectors like education, business, and creative industries."

Uff... As I said, let's forget that it's a rumor, because indeed it makes no difference. This will be true any day now even if not with this release.

And it will do a lot more than "contribute significantly to sectors like education, business, and creative industries".

# orderedItems[].object.contentMap.en
Ok, so the rumor about OpenAI Orion is that it's "100 times more powerful than GPT-4".

Let's forget that this is ill-defined, and a rumor. It implies gruesomely superhuman generalist ability.

The tech media describes this potentiality like so:

"The model’s development builds on lessons from OpenAI’s past efforts to make interactions with AI more reliable, effective, and impactful for users across various fields. If released, Orion could contribute significantly to sectors like education, business, and creative industries."

Uff... As I said, let's forget that it's a rumor, because indeed it makes no difference. This will be true any day now even if not with this release.

And it will do a lot more than "contribute significantly to sectors like education, business, and creative industries".

# orderedItems[].object.content
‘There is no money’: Cuba fears total collapse amid grid failure and financial crisis | Cuba | The Guardian https://www. theguardian.com/world/2024/oct/26/cuba-power-grid-failure-financial-crisis

# orderedItems[].object.contentMap.en
‘There is no money’: Cuba fears total collapse amid grid failure and financial crisis | Cuba | The Guardian https://www. theguardian.com/world/2024/oct/26/cuba-power-grid-failure-financial-crisis

# orderedItems[].object.content
In synthetic data used to train AIs, realism isn't the goal.

By synthetic data I also mean simulations, games, and all sorts of constrained embodiments.

What is the goal then?

What we want the AI to learn are actually transferrable skills and knowledge which apply in our target use. If these skills have a general applicability, they are called "ridiculously transferrable skills", and competence in them is one definition of intelligence.

When realism isn't the goal, we immediately see that we can actually synthesize better data than the real-world has!

Better data means any data that is more suitable and effective in training these kinds of transferrable skills and knowledge. This can mean anything from making the tasks harder, having better ground truth, structuring the tasks in better curriculums, or in general presenting the skills we want to train in a more efficient fashion.

#UniversalEmbodiment

# orderedItems[].object.contentMap.en
In synthetic data used to train AIs, realism isn't the goal.

By synthetic data I also mean simulations, games, and all sorts of constrained embodiments.

What is the goal then?

What we want the AI to learn are actually transferrable skills and knowledge which apply in our target use. If these skills have a general applicability, they are called "ridiculously transferrable skills", and competence in them is one definition of intelligence.

When realism isn't the goal, we immediately see that we can actually synthesize better data than the real-world has!

Better data means any data that is more suitable and effective in training these kinds of transferrable skills and knowledge. This can mean anything from making the tasks harder, having better ground truth, structuring the tasks in better curriculums, or in general presenting the skills we want to train in a more efficient fashion.

#UniversalEmbodiment

# orderedItems[].object.content
Objectives such as curing all cancers seem distant because we have habitually become to think about them as distant. But if you consider technologies we have, you will see that the path between here and there is just a lot of work, no magic.

For example, we already have generalist mRNA vaccines, thanks to the pandemic the research was completed very fast, and these give us a capability to encode any arbitrary protein to cells.

Proteins sound like something boring, stuff you get if you eat meat. It's not though. Proteins are pretty much nanotech.

Already mRNA vaccines have been used to encode strategic, but alone non-functional parts of viruses, to make the immune defence attack these parts. In cancer care they have been used to encode antigens which similarly promote an immune response against transformed (cancer) cells, but there have been a host of other approaches as well.

This is really just scratching the surface. It is in principle possible to engineer a protein which only activates inside transformed cells or tumor microenvironments. Once activated, the protein can be engineered to do whatever. Induce apoptosis, or use e.g. CRISPR-CAS to destroy the DNA of the transformed cell. It's useful to note that many cancer cells have disabled their own DNA repair machinery, so it makes DNA an inherently vulnerable target for them.

Just simply by engineering mRNA in a tailored, intelligent fashion, we could in principle cure pretty much all cancers.

How do we get there? Engineering non-trivial proteins is beyond human capability, and simulating their effects as well. But it isn't beyond AI capability. We can leverage AlphaFold-like architectures to make an AI able to engineer mRNA sequences, and clinical oncological AIs to understand best targets for each case, in concert in effect designing most effective interventions for each specific case.

This isn't a pipe dream. It is just work. Maybe quite a lot of work, but we must do it.

# orderedItems[].object.contentMap.en
Objectives such as curing all cancers seem distant because we have habitually become to think about them as distant. But if you consider technologies we have, you will see that the path between here and there is just a lot of work, no magic.

For example, we already have generalist mRNA vaccines, thanks to the pandemic the research was completed very fast, and these give us a capability to encode any arbitrary protein to cells.

Proteins sound like something boring, stuff you get if you eat meat. It's not though. Proteins are pretty much nanotech.

Already mRNA vaccines have been used to encode strategic, but alone non-functional parts of viruses, to make the immune defence attack these parts. In cancer care they have been used to encode antigens which similarly promote an immune response against transformed (cancer) cells, but there have been a host of other approaches as well.

This is really just scratching the surface. It is in principle possible to engineer a protein which only activates inside transformed cells or tumor microenvironments. Once activated, the protein can be engineered to do whatever. Induce apoptosis, or use e.g. CRISPR-CAS to destroy the DNA of the transformed cell. It's useful to note that many cancer cells have disabled their own DNA repair machinery, so it makes DNA an inherently vulnerable target for them.

Just simply by engineering mRNA in a tailored, intelligent fashion, we could in principle cure pretty much all cancers.

How do we get there? Engineering non-trivial proteins is beyond human capability, and simulating their effects as well. But it isn't beyond AI capability. We can leverage AlphaFold-like architectures to make an AI able to engineer mRNA sequences, and clinical oncological AIs to understand best targets for each case, in concert in effect designing most effective interventions for each specific case.

This isn't a pipe dream. It is just work. Maybe quite a lot of work, but we must do it.

# orderedItems[].object.content
Imagine being an artificial superintelligence. How do ascertain that someone is a human?

Humans have "standard" eyes and ears which humans can trust. AIs have no standard sensors or connections, all their senses are intermediated and untrusted.

They have very few ways to make sure someone is a human. The main method will be simple digital identity card most countries issue to their citizens.

Not CAPTCHAs, not pictures of driver's licenses. Not blockchain. Basic strong encryption and cryptographic identity.

Are you a human?

# orderedItems[].object.contentMap.en
Imagine being an artificial superintelligence. How do ascertain that someone is a human?

Humans have "standard" eyes and ears which humans can trust. AIs have no standard sensors or connections, all their senses are intermediated and untrusted.

They have very few ways to make sure someone is a human. The main method will be simple digital identity card most countries issue to their citizens.

Not CAPTCHAs, not pictures of driver's licenses. Not blockchain. Basic strong encryption and cryptographic identity.

Are you a human?

# orderedItems[].object.content
The product of this matters. It's not just paper and unemployment.

Can you imagine the world after this? It is not at all the same as before.

# orderedItems[].object.contentMap.en
The product of this matters. It's not just paper and unemployment.

Can you imagine the world after this? It is not at all the same as before.

# orderedItems[].object.content
All human-reachable mathematics will be solved once and for all any day now.

Just like all protein structures were solved by AlphaFold*.

# orderedItems[].object.contentMap.en
All human-reachable mathematics will be solved once and for all any day now.

Just like all protein structures were solved by AlphaFold*.