I claim that "AIs will be good at using APIs before they are good at computer use" is true but mostly irrelevant to existential risk. I agree that it's convenient for some current stuff around AI reliability, but I struggle to see a story where it reduces risk from misaligned superintelligence. If you want to make some broader claim of like "unexpected things happen and they're sometimes good for safety", sure. Are you trying to make a stronger claim vis a vis x-risk?
On a different note:
> Granular permissions, human-readable audit logs, and explicit action approval workflows could become the norm without safety regulations, but because they're what developers and organizations want and use. For example, Claude and OpenAI’s API options have taken off much faster than OpenAI’s Operator, their AI agent that uses a virtual browser that still sits behind a $200 a month paywall.
I don't think that example is a fair comparison. The Chat APIs have been around for like 2+ years, Operator for 6 months, and they're totally different use cases, which I think is basically what explains the difference in use. I don't disagree with the claim 'Granular permissions, human-readable audit logs, and explicit action approval workflows will be desired by customers', but this example is just not very relevant.
Thanks Aaron! On the relevance to x-risk: there's some chance that companies and people who use AI will have a strong preference for these more granular APIs. If that's the case, where the norm becomes that, advanced AI would in some way be held back in what it can do. It's an oversight mechanism that I think is good news. Whether the claim goes all the way to superintelligence and to every instance of superintelligence, I don't think it solves that claim. I.e. even if only-interacting-with-world-via-API limits what actions superintelligence can do, I think it's unlikely that every single AI system that people decide to run would be run like that.
I agree about the Operator being so new is not a direct comparison. But it's an indicator of some chance that virtual browser agents take a long time to become dependable i.e. how self-driving cars are taking longer than the developers expected.
I'm more wanting to make a claim about the direction, and that there's some good news that I think is just not being written about, which is surprising.
Cool, thanks for clarifying! I agree with the direction of the claim and with your comment that "I think it's unlikely that every single AI system that people decide to run would be run like that." And think it also applies to many of the 'good signs' we might see in the next couple years. e.g., 'Chain of Thought preserves some human understandability' relies on developers continuing to use it.
I claim that "AIs will be good at using APIs before they are good at computer use" is true but mostly irrelevant to existential risk. I agree that it's convenient for some current stuff around AI reliability, but I struggle to see a story where it reduces risk from misaligned superintelligence. If you want to make some broader claim of like "unexpected things happen and they're sometimes good for safety", sure. Are you trying to make a stronger claim vis a vis x-risk?
On a different note:
> Granular permissions, human-readable audit logs, and explicit action approval workflows could become the norm without safety regulations, but because they're what developers and organizations want and use. For example, Claude and OpenAI’s API options have taken off much faster than OpenAI’s Operator, their AI agent that uses a virtual browser that still sits behind a $200 a month paywall.
I don't think that example is a fair comparison. The Chat APIs have been around for like 2+ years, Operator for 6 months, and they're totally different use cases, which I think is basically what explains the difference in use. I don't disagree with the claim 'Granular permissions, human-readable audit logs, and explicit action approval workflows will be desired by customers', but this example is just not very relevant.
Thanks Aaron! On the relevance to x-risk: there's some chance that companies and people who use AI will have a strong preference for these more granular APIs. If that's the case, where the norm becomes that, advanced AI would in some way be held back in what it can do. It's an oversight mechanism that I think is good news. Whether the claim goes all the way to superintelligence and to every instance of superintelligence, I don't think it solves that claim. I.e. even if only-interacting-with-world-via-API limits what actions superintelligence can do, I think it's unlikely that every single AI system that people decide to run would be run like that.
I agree about the Operator being so new is not a direct comparison. But it's an indicator of some chance that virtual browser agents take a long time to become dependable i.e. how self-driving cars are taking longer than the developers expected.
I'm more wanting to make a claim about the direction, and that there's some good news that I think is just not being written about, which is surprising.
Cool, thanks for clarifying! I agree with the direction of the claim and with your comment that "I think it's unlikely that every single AI system that people decide to run would be run like that." And think it also applies to many of the 'good signs' we might see in the next couple years. e.g., 'Chain of Thought preserves some human understandability' relies on developers continuing to use it.
I wish the term superintellenge would go away. It’s this trendy cocktail party term . I read Bostroms book and it’s pretty much just garbage