Arty was a UQ DECO3801 capstone project for digital illustrators. The product goal was not to replace the artist; it was to reduce the tedious parts of a concept workflow: finding references, organising ideas, and producing fast visual starting points while keeping creative control with the person drawing.
Features
- Reference search - search Pinterest-style image sources from text, uploaded images, or a combination of both.
- Image understanding - use BLIP captions from uploaded images to improve search queries.
- AI visualisation - generate starting-point imagery through Stable Diffusion and ControlNet.
- Creative workspace - keep references and generated concepts beside the artist's canvas.
- Account/history support - persist users, search history, and generated workflow data with PostgreSQL.
My role
I was the Machine Learning Developer on a six-person team. My main focus was the AI backend: integrating Stable Diffusion, wiring ControlNet options, exposing generation endpoints through Flask, and connecting image-captioning/search flows so artists could move from rough visual references to generated concepts.
Technical notes
The Flask backend loads BLIP for image captions, wraps Stable Diffusion WebUI/ControlNet through a Python backbone, and supports both text-to-image and inpainting-style upload flows. The search endpoint can combine a typed query with a caption generated from an uploaded image, which made the reference search feel more visual than a normal keyword box.
The project also includes database routes for user accounts and search history, plus a React frontend with pages for home, search, upload, profile, terms, privacy, and about content. For a student capstone, the interesting challenge was not any single model call - it was turning several brittle AI and web pieces into one understandable artist workflow.
The team
| Name | Role |
|---|---|
| William Mercado | Team Leader |
| Brian Zhang | Machine Learning Developer |
| Hongyingzi Lu | Design / Front-End |
| Shan Liu | Design / Front-End |
| Yang Xiao | Back-End Developer |
| Ryuto Hisamoto | Back-End Developer |