LoRa in AI image generators

This stands for Low Rank adaptation. In AI terms, this is a small add-on to the model that makes it do specific things. It is usually used in the context of stable diffusion. A large model takes very long to train with significant resources. However, sometimes, you need specific details – for example you could have a model that needs to do better image generation of the hands, in which case you train a small subset with hands and that is the LoRa that will be used specifically for hands. Remember, LoRa does not do the full model but just freezes weights and parameters. Then it adds matrices which are applied to new inputs to get specific results in the context that is needed.

LoRa is used sometimes to create images in a specific style.


In the picture above that image was generated with an image generator in the drawing style. Whereas in the fantasy or other styles it looks very different with the same prompt.

Similar Posts

  • LoRa adaptors

    LoRa in the IOT community are the low power long range wireless standards devices that are utilized to send a signal over large distances. However, in the AI field LoRa implies Low rank adaptation – which is relatively more efficient compute way to fine tune pretrained models (LLM, vision transformers, diffusion models). With a pretrained…

  • |

    Virtual cell

    Hani Goodrazi – Arc institute has been working on virtual cell. Drugs fail due to overfit experimental models, You need screen drugs with better models of human biology. Geneva is platfrom that brings tumor models into perturbation model – that is a transcriptomics assay that deconvolves into effect. Take multiple cell line and then treat…

  • AI in subjective domains

    I am exploring how AI can scale expert feedback in subjective domains with a hobby project. After visiting art museums around the world, I kept coming back to a familiar challenge: how do you consistently evaluate something inherently subjective? After having created medical devices in pain, the idea was worth exploring. It’s a question that…

  • | |

    Functional prediction of microbial sequences

    Even with E.coli and M.tuberculosis we only know ½ of them. Can you use ML model to define function: as natural language, or molecular interaction or chemical reaction. Function as molecular interaction: protein – protein interaction. Genomics: Learn association between genes (just like words). It is called gLM2. A multi-modal single residue resolution gLM. GLM2…

  • |

    AI for drug discovery

    There are many companies that promise AI for drug discovery such as ChatGPT from OpenAI, Anthropic’s AI suite and Perplexity and there are more companies coming up too. One specific company that is unusual is that it seems to be promising much more deeper understanding of the drug discovery pathways then the others. It can…