LoRa in AI image generators

This stands for Low Rank adaptation. In AI terms, this is a small add-on to the model that makes it do specific things. It is usually used in the context of stable diffusion. A large model takes very long to train with significant resources. However, sometimes, you need specific details – for example you could have a model that needs to do better image generation of the hands, in which case you train a small subset with hands and that is the LoRa that will be used specifically for hands. Remember, LoRa does not do the full model but just freezes weights and parameters. Then it adds matrices which are applied to new inputs to get specific results in the context that is needed.

LoRa is used sometimes to create images in a specific style.


In the picture above that image was generated with an image generator in the drawing style. Whereas in the fantasy or other styles it looks very different with the same prompt.

Similar Posts

  • | | |

    Biotech companies

    Small Biotechs: Diagonal Tx: Clustering antibodies that mimic the action of the ligand and bypass the need for the ligand and receptor. This mutation that is created makes standard AI models not useful and so need a new method. This restores new ALK1 signaling in Hereditary Hemorrhagic Telangiectasia. It also treates LoF mutations in ALK1…

  • Observability

    Observability is important for AI and AI tools. It is the ability to monitor them for token usage, response quality and model drift. Typically, an AI system is monitored through logs, traces and metrics but an AI system on AI agent may need other metrics. Troubleshooting a complex AI system that produces its output probabilistically…

  • |

    Fitness landscape for antibodies (Flab2)

    There have been many AI models to predict the structure of proteins, especially antibodies. There are several key aspects to developing antibodies as drugs beyond binding to the target of interest. These are : thermostability, expression, aggregation, binding affinity, pharmacokinetics, polyreactivity and immunogenicity. There have been many models available such as IgLM, ProGen2, Chai-1, ESM2,…

  • AI in subjective domains

    I am exploring how AI can scale expert feedback in subjective domains with a hobby project. After visiting art museums around the world, I kept coming back to a familiar challenge: how do you consistently evaluate something inherently subjective? After having created medical devices in pain, the idea was worth exploring. It’s a question that…

  • Explainable AI

    A very traditional problem solving method is the following: given a set of features or variables, can we understand the features to form a conclusion. This could be something like a treatment strategy wherein the strategy is built on a series of data and then ingesting the data helps make a conclusion. However, an equally…