cm0002@lemmy.world to Artificial Intelligence@lemmy.worldEnglish · 1 month agoLLMs Can Think While Idle: Researchers from Letta and UC Berkeley Introduce ‘Sleep-Time Compute’ to Slash Inference Costs and Boost Accuracy Without Sacrificing Latencywww.marktechpost.comexternal-linkmessage-square3linkfedilinkarrow-up12arrow-down10
arrow-up12arrow-down1external-linkLLMs Can Think While Idle: Researchers from Letta and UC Berkeley Introduce ‘Sleep-Time Compute’ to Slash Inference Costs and Boost Accuracy Without Sacrificing Latencywww.marktechpost.comcm0002@lemmy.world to Artificial Intelligence@lemmy.worldEnglish · 1 month agomessage-square3linkfedilink
minus-squarejwmgregory@lemmy.dbzer0.comlinkfedilinkEnglisharrow-up1·22 days agoi mean… your brain essentially does this it’s just that compute and memory are one system and it is as physically optimized as possible in brain systems. this strategy is less stupid than it sounds if you abandon von neumann purism imo it can be more efficient than just waiting for the input and inferring once based on that… you are an example of this in real life.
minus-squarevrighter@discuss.tchncs.delinkfedilinkEnglisharrow-up1·22 days agofaster != more efficient. And you cannot compare brains to computers. Speculative execution improves speed in the cpu, at the cost of efficiency
i mean…
your brain essentially does this it’s just that compute and memory are one system and it is as physically optimized as possible in brain systems.
this strategy is less stupid than it sounds if you abandon von neumann purism imo
it can be more efficient than just waiting for the input and inferring once based on that… you are an example of this in real life.
faster != more efficient. And you cannot compare brains to computers. Speculative execution improves speed in the cpu, at the cost of efficiency