KeysUsageBilling & Credits
Guides
HomeAPIStatus

Video Generation

🚀

Looking for the latest Luma API?

Create and manage API access in the Luma API Platform. For current API guides and reference, visit the Luma Agents API documentation.

Installation

npm install lumaai

https://www.npmjs.com/package/lumaai

Authentication

  1. Get a key from https://platform.lumalabs.ai/

In JavaScript, you can pass the API key using the authToken parameter when creating the client.

const { LumaAI } = require('lumaai');

const client = new LumaAI({
  authToken: process.env.LUMAAI_API_KEY
});

Models

namemodel param
Ray 2 Flashray-flash-2
Ray 2ray-2

Polling for Generation Status and Retrieving Video

In JavaScript, use async/await for handling the polling.

npm install node-fetch
const fetch = require('node-fetch');
const fs = require('fs');
const { LumaAI } = require('lumaai');

const client = new LumaAI({ authToken: process.env.LUMAAI_API_KEY });

async function generateVideo() {
    let generation = await client.generations.create({
        prompt: "A teddy bear in sunglasses playing electric guitar and dancing",
        model: "ray-2"
    });

    let completed = false;

    while (!completed) {
        generation = await client.generations.get(generation.id);

        if (generation.state === "completed") {
            completed = true;
        } else if (generation.state === "failed") {
            throw new Error(`Generation failed: ${generation.failure_reason}`);
        } else {
            console.log("Dreaming...");
            await new Promise(r => setTimeout(r, 3000)); // Wait for 3 seconds
        }
    }

    const videoUrl = generation.assets.video;

    const response = await fetch(videoUrl);
    const fileStream = fs.createWriteStream(`${generation.id}.mp4`);
    await new Promise((resolve, reject) => {
        response.body.pipe(fileStream);
        response.body.on('error', reject);
        fileStream.on('finish', resolve);
    });

    console.log(`File downloaded as ${generation.id}.mp4`);
}

generateVideo();

Ray 2 Text to Video

const generation = await client.generations.create({
  prompt: "A teddy bear in sunglasses playing electric guitar and dancing",
  model: "ray-2",
  resolution: "720p",
  duration: "5s"
});

Resolution can be 540p, 720p, 1080, 4k

Ray 2 Image to Video

const generation = await client.generations.create({
  prompt: "Low-angle shot of a majestic tiger prowling through a snowy landscape, leaving paw prints on the white blanket",
  model: "ray-2",
  keyframes: {
    frame0: {
      type: "image",
      url: "https://your-image-url.com/start-frame.jpg"
    }
  }
});

How to use concepts

const generation = await client.generations.create({
  prompt: "a car",
  model: "ray-2",
  resolution: "720p",
  duration: "5s",
  concepts: [
    {
    	"key": "bolt_cam"
    }
  ]
});

We have this new /concepts/list api one can hit to get list of concepts available

https://api.lumalabs.ai/dream-machine/v1/generations/concepts/list

Text to Video

Text to Video Generation Example

const generation = await client.generations.create({
  prompt: "A teddy bear in sunglasses playing electric guitar and dancing"
  model: "ray-2"
});

Image to Video

With start frame

const generation = await client.generations.create({
  prompt: "Low-angle shot of a majestic tiger prowling through a snowy landscape, leaving paw prints on the white blanket",
  model: "ray-2"
  keyframes: {
    frame0: {
      type: "image",
      url: "https://your-image-url.com/start-frame.jpg"
    }
  }
});

With start frame, loop

const generation = await client.generations.create({
  prompt: "Low-angle shot of a majestic tiger prowling through a snowy landscape, leaving paw prints on the white blanket",
  model: "ray-2"
  keyframes: {
    frame0: {
      type: "image",
      url: "https://your-image-url.com/start-frame.jpg"
    }
  },
  loop: true,
});

With end frame, loop

const generation = await client.generations.create({
  prompt: "Low-angle shot of a majestic tiger prowling through a snowy landscape, leaving paw prints on the white blanket",
  model: "ray-2"
  keyframes: {
    frame1: {
      type: "image",
      url: "https://your-image-url.com/start-frame.jpg"
    }
  },
  loop: true,
});

Extend video

Extending Video

Extend is currently supported only for generated videos. Please make sure the generation is in completed state before passing it

const generation = await client.generations.create({
  prompt: "A teddy bear in sunglasses playing electric guitar and dancing",
  model: "ray-2"
  keyframes: {
    frame0: {
      type: "generation",
      id: "d1968551-6113-4b46-b567-09210c2e79b0"
    }
  }
});

Reverse Extend Video

Extend is currently supported only for generated videos. Please make sure the generation is in completed state before passing it

const generation = await client.generations.create({
  prompt: "A teddy bear in sunglasses playing electric guitar and dancing",
  model: "ray-2"
  keyframes: {
    frame1: {
      type: "generation",
      id: "d1968551-6113-4b46-b567-09210c2e79b0"
    }
  }
});

Interpolating Between Two Generated Videos

Interpolate is currently supported only for generated videos. Please make sure the generation is in completed state before passing it

const generation = await client.generations.create({
  prompt: "A teddy bear in sunglasses playing electric guitar and dancing",
  model: "ray-2"
  keyframes: {
    frame0: {
      type: "generation",
      id: "d312d37a-7ff4-49f2-94f8-218f3fe2a4bd"
    },
    frame1: {
      type: "generation",
      id: "d1968551-6113-4b46-b567-09210c2e79b0"
    }
  }
});

Generations

Get generation with id

const generation = await client.generations.get("4fac0ef4-b336-45bf-a5dc-6de436cfbd62");

List all generations

const generation = await client.generations.list();

Delete generation

const generation = await client.generations.delete("4fac0ef4-b336-45bf-a5dc-6de436cfbd62");

Camera Motions

Get all supported camera motions

const cameras = await client.generations.cameraMotion.list();

Docs

https://www.npmjs.com/package/lumaai

Please see API docs for API reference.


How to get a callback when generation has an update

  • It will get status updates (dreaming/completed/failed)
  • It will also get the video url as part of it when completed
  • It's a POST endpoint you can pass, and request body will have the generation object in it
  • It expected to be called multiple times for a status
  • If the endpoint returns a status code other than 200, it will be retried max 3 times with 100ms delay and the request has a 5s timeout

example

const generation = await client.generations.create({
  prompt: "A teddy bear in sunglasses playing electric guitar and dancing",
  callback_url: "<your_api_endpoint_here>"
});