So I've done a little bit of testing. When I loaded up the editor, I expected it to communicate with the installed binaries for my large language models, or possibly use a BYOK (bring your own key) approach. But once I understood that it's driven from the command line and you see the changes happen in the editor that gets spun up, I had no problems.
I've been experimenting a fair amount with LLMs driving procedural graphics myself, although my experiments have been mostly limited to relatively low-resolution stuff, such as EGA adventure game graphics.
So I decided to put Klad through a relatively hard test: I took a reference picture of an illustrated squirrel and asked it to recreate it, then set up the rigging for a basic walk cycle. This was using Fable 5.2 high.
Then I did the same thing, but with a more traditional approach, using standard generative models to manually create a walk cycle. This was all done relatively quickly, so bear that in mind.
Here are the results. I still think these are early days, and LLM-driven graphics are only going to improve.
And there are obvious advantages to a procedural system with rigging: Much easier to maintain consistency, palette control, and the ability to make precise alterations without having to deal with any kind of conventional in-painting nonsense.
I have been meaning to find/build a tool that would sit in between the pre-production assets of a movie clip (script, characters, locations, prop assets) and the moment you press generate.
I envisioned it as some sort of a flipbook that would allow me to *cheaply* get a low-res idea of what the 'generate' click would get me, without having to burn movie making tokens.
This may not be exactly it, but it nudged me into a direction. Thanks!
I will give it a try and report back.
mmvvaa
I had a similar idea, but for photo manipulation.
I like photography, but I'm relatively poor on the photo editing. I was hoping to build an MCP server around Gimp/Darktable/RawTherapee. The idea would be that the user would describe changes he wanted to see in the image (brighten up the shirt), and the LLM would pick appropriate tools to select the region and make the edit.
At the time I looked into it, it didn't seem that Darktable/RawTherapee provided APIs to hook into. Then life got busy.
BeetleB
Sorry if I missed it, but how is it different from having the "agent" write SVG code?
theanonymousone
I have been trying to build something similar as well. Is there a browser version to try or an MCP?
hvvivek
The project looks really cool. Do you have an average measure of the time taken for a generation and the number of tool calls?
rchrp
Hmm this is interesting, perhaps you could pair it with image gen and add code to seperate the layers instead of individual brush strokes. Could be very token costly. Will try though
comments (10)
ramigb
https://github.com/imjasonh/playground/pull/715
ImJasonH
So I've done a little bit of testing. When I loaded up the editor, I expected it to communicate with the installed binaries for my large language models, or possibly use a BYOK (bring your own key) approach. But once I understood that it's driven from the command line and you see the changes happen in the editor that gets spun up, I had no problems.
I've been experimenting a fair amount with LLMs driving procedural graphics myself, although my experiments have been mostly limited to relatively low-resolution stuff, such as EGA adventure game graphics.
So I decided to put Klad through a relatively hard test: I took a reference picture of an illustrated squirrel and asked it to recreate it, then set up the rigging for a basic walk cycle. This was using Fable 5.2 high.
Then I did the same thing, but with a more traditional approach, using standard generative models to manually create a walk cycle. This was all done relatively quickly, so bear that in mind.
Here are the results. I still think these are early days, and LLM-driven graphics are only going to improve.
And there are obvious advantages to a procedural system with rigging: Much easier to maintain consistency, palette control, and the ability to make precise alterations without having to deal with any kind of conventional in-painting nonsense.
https://imgpb.com/vJFpP
vunderba
I envisioned it as some sort of a flipbook that would allow me to *cheaply* get a low-res idea of what the 'generate' click would get me, without having to burn movie making tokens.
This may not be exactly it, but it nudged me into a direction. Thanks!
I will give it a try and report back.
mmvvaa
I like photography, but I'm relatively poor on the photo editing. I was hoping to build an MCP server around Gimp/Darktable/RawTherapee. The idea would be that the user would describe changes he wanted to see in the image (brighten up the shirt), and the LLM would pick appropriate tools to select the region and make the edit.
At the time I looked into it, it didn't seem that Darktable/RawTherapee provided APIs to hook into. Then life got busy.
BeetleB
theanonymousone
hvvivek
rchrp
arjgoel30
geooff_