• BakedCatboy@lemmy.ml
    link
    fedilink
    English
    arrow-up
    0
    ·
    24 days ago

    If you have a lot of RAM, you can run small models slowly on the CPU. Your integrated graphics I would guess won’t fit anything useful in it’s vram, so if you really want to run something locally, getting some extra sticks of RAM is probably your cheapest option.

    I have 64G and I run 8-14b models. 32b is pushing it (it’s just really slow)

    • Lucy :3@feddit.org
      link
      fedilink
      arrow-up
      1
      ·
      24 days ago

      Don’t iGPUs use the RAM as VRAM directly? You’d only need to configure how much in the BIOS (eg. by default it uses 1.5GB of 8GB or smth and you can set it to 6/8GB)