pic2recipe

What I added this devlog:

Recipe Sharing
Added a share button on the recipe detail page next to the favorite heart. Uses React Native’s built in Share API so no new libraries were needed. Formats the recipe into a readable text block with the title, cook time, difficulty, ingredients list, and numbered steps. Opens the phone’s share sheet so users can send it through text, email, or any other app.
“I Made This” Button
Added a button that shows up at the bottom of the instructions tab. When you press it, it looks at the recipe’s ingredients, checks which ones match what’s in your pantry, and removes those from the pantry. Then it sends you back to the home screen.

To make this work I had to:

• Pass the user’s pantry ingredients from the results page through RecipeCard into the recipe detail page as a route parameter
• Add a removeIngByTitle function to the database operations since the existing remove function only worked with IDs
• Update InstructionOption to accept and pass down the ingredient lists
• Create a new MadeRecipeButton component that handles the filtering and removal logic

Favorites Feature

What I Built:
Added a favorites tab where users can save recipes they like. There’s a heart icon on every recipe card and on the recipe detail page. Tapping it saves the recipe locally. The favorites tab shows all saved recipes with a count at the top.

How It Works
Favorites are stored in WatermelonDB, same as the pantry ingredients. I added a new favorites table to the schema with columns for recipe ID, title, ingredients, directions, cook time, difficulty, and equipment. Since WatermelonDB doesn’t support array columns, I store arrays as JSON strings and parse them when reading.
The heart icon is its own component called FavoriteButton. It checks the database on mount to see if the recipe is already saved, and fills the heart if it is. Tapping it adds or removes the recipe.
The favorites tab uses withObservables to reactively display saved recipes. When you favorite or unfavorite something, the list updates right away without needing to refresh.
Problems I Ran Into

Heart not filled on recipe detail page - I was passing the wrong object to FavoriteButton. The recipe from useRecipeDetail doesn’t have an id field, so the favorite check always failed. Fixed by passing the manually built recipe object that includes the ID.
“0 min” instead of “Unknown” - WatermelonDB returns 0 for empty number columns instead of null. The ?? operator doesn’t catch 0 because it only handles null and undefined. Switched to || which treats 0 as falsy.
“Uses your: “ with nothing after it - The favorites list was passing an empty ingredients array to RecipeCard. Fixed by also observing the ingredients table in FavoritesList so it knows what’s in the pantry.
Heart positioning - The heart icon would sometimes go off the card depending on title length. Fixed by using absolute positioning in the top right corner with pr-12 on the title so they don’t overlap.
Tests breaking after merge - The ingUsed.test.ts test started failing because RecipeCard now imports FavoriteButton, which imports the database, which tries to start SQLiteAdapter. SQLiteAdapter is a native module that doesn’t exist in Jest. Fixed by mocking the database module in the test file.
Schema migration - Had to bump the schema version from 2 to 3 and write a migration file so existing installs don’t crash. Also had to clear app data on my phone during development because the old database didn’t have the favorites table.

VIDEO DEMO: https://youtube.com/shorts/t7tEtORol2o?feature=share

• Fixed the ingredient scanner not working in the apk (had to add a secret in eas)
• Made the find recipes button smaller so it’s easier to delete ingredients (used to be covered)
• added a label on the scan tab for better UX
• Added ability to scan receipts too in the scan tab
• Integrated Sentry (error reporting software)
  No ios support unfortunately because it costs $100

Finalized app before shipping:

• added an image icon (not just the default app)
• changed app name
• added readme

ALMOST READY TO SHIP!!

In this devlog, i’ve:

• added frontend tests with the Jest testing framework, and integrated a continuous integration pipeline using Github Actions to automate testing on push/pull request
• added some more small ui changes like margins/padding, only showing the “Add to pantry” button when Gemini API actually identifies ingredients, and showing the user an error screen when they press the “Find Recipes” button without having any ingredients.
• turned the app into an APK using EAS and right now i’m doing some last minute testing before shipping.

ONE MAJOR SETBACK:

• My demo will only work on android unfortunately because it’ll cost $100 to launch the app on ios (i’m too poor)
• also i have an android so i can’t test on ios even if i could pay the $100

NEXT:

• SHIPP

I implemented a continuous integration pipeline using github actions to automate the backend tests. This ensures that new features that i add won’t break old features.

again this doesn’t change anything in the actual project so the image is just the test cases passing on github.

I did a lot of learning with docker and containerized the backend + ngrok tunnel i use to send requests from my phone to the server. I also created tests for the backend, testing the service (getting reccomendations), repository (querying the db based on query embedding values), and routers (making sure that valid ingredients would work and invalid ones wouldn’t). I also created a separate mock test db in docker to do this testing. I learned a ton about pytest and how tests should be done.

Since this stuff doesn’t actually show on my ui, i didn’t really know what to screenshot but there are my running containers and tests working.

Trained YOLO11n on two different food datasets on Kaggle. Both models could locate objects on-device but couldn’t classify them correctly — always predicted the last class index regardless of input
Debugged for days. Tried every combination of TFLite export settings (float16/float32, with/without NMS, different input normalizations). None worked. YOLO’s classification head breaks during TFLite conversion. Found a GitHub issue confirming this is a known problem with no fix

Gave up on on-device ML. Switched to Gemini 2.5 Flash API -> send photo, get back ingredient labels + bounding box coordinates. Took 30 minutes to implement vs days of YOLO debugging (are we serious)

I fixed the bounding box logic. But the model I trained is pretty bad and innacurate (see attached photo, it detected an apple as a lemon) so i’m currently training another model in hopes that that one is more accurate.

I also still have to:

• make it so once the user presses the “add to pantry” button, it’ll actually add to the db
• fix some race condition errors with the model loading then running inference with the capture button

But after fixing those and some polishing, i’m basically done the MVP for this project!

I’ve made a ton of small UI fixes and changes for a better user experience. for example, adding scrolling to the ingredients list, loading screens for everything. i’ve also started on implementing the ml model to detect ingredients. I’ve added the camera, trained an ml model, did a ton of debugging, and now have code that runs inference on it to detect ingredients (this works), but right now the bounding box logic i’m trying to implement isn’t working so i’m debugging that right now.

The results screen now uses the ingredients inside the ingredients db instead of being hardcoded. And i also fixed a bug with the options on an individual recipe screen not working

added watermelondb, which is a database which uses sqlite but has lazy loading. I first had to include it in my config files, then i created the Ingredient model, table schema, and functions to add and remove ingredients from the db. And finally, i added a “+” button on the main page to allow users to add and remove ingredients from the db.

added equipment, time, and difficulty to the actual ui. Also wrote a little script to find what ingredients the recipe uses that the user has. Also fixed some ts errors.

I added cook time, equipment and difficulty to the database for a better user experience using regexs to scan the directions and ingredients for key words. I first created the test_parsers.py file to test whether my parsers would work, then in enrich_recipes.py, I used the parsers to load new columns in the db. Then I modified the sqlalchemy schemas and pydantic models in the backend to account for the new columns

made it so the frontend will actually display the recipe info based on the ingredients given (those are hard coded for now) on the results page. And on the individual recipes page, it now displays the recipes info like directions, ingredients.

so i didn’t find out about Flavortown until I was already somewhat progressed in this project, hence the 26h devlog. I did create a log of my work though so here it is all at once:

March 14, 2026

• Initialized repo with project scaffolding: .gitignore, LICENSE (MIT), README, placeholder CI workflow
• Added initial Expo frontend (app.json) and backend requirements.txt
• Included a TFLite model binary for early camera/ML experimentation
• Cleaned up tracking — removed package.json and package-lock.json from version control

March 20, 2026

• Built the data pipeline for loading recipe embeddings into PostgreSQL
• Created init.sql to set up the recipes table with pgvector (vector(768)) column
• Wrote load_recipes.py — reads RecipeNLG CSV + a numpy memmap of pre-computed embeddings, batch-inserts into DB (512 rows/batch)
• Added test_query.py to verify cosine similarity search against the DB using sentence-transformers/all-mpnet-base-v2
• Removed the TFLite binary and old plan.txt from repo

March 26, 2026

• Scaffolded the full backend package structure: api/, repositories/, services/, schemas/, tests/
• Defined SQLAlchemy ORM model (Recipe) mapping to the recipes table with pgvector Vector(768)
• Created Pydantic schemas: IngredientsRequest (input) and RecipeResponse (output with similarity score)
• Set up FastAPI app in main.py with CORS middleware and lifespan-based model loading (SentenceTransformer)
• Added config.py using pydantic-settings to pull DB and model config from .env
• Stubbed out the POST /recipe route, repo, and service layer (empty files)

March 27, 2026

• Implemented the core querying logic end-to-end
• dependencies.py — yield-based SQLAlchemy session factory for FastAPI’s Depends()
• recipe_repo.py — pgvector cosine distance query, returns top-k results with similarity scores
• recommendation.py — service layer that encodes ingredient text into a vector and passes it to the repo

March 28, 2026

• Wired everything together to complete the backend API
• Connected the POST /recipe router to the service layer with dependency injection (DB session + ML model)
• Extracted ml_model dict into a separate ml.py module to resolve circular import between main.py and the router
• Added error handling (HTTPException 404) on the recipe endpoint

April 3, 2026

• Built the frontend UI shell in React Native with Expo Router and NativeWind (Tailwind v3)
• Set up file-based routing: (tabs) layout with Kitchen Hub, Scan, and Results screens + recipe/[id].tsx
• Created reusable components: RecipeCard (results list item) and IngredientItem
• Configured Babel, Metro, Tailwind, TypeScript, ESLint, and NativeWind
• Placeholder recipe/[id].tsx page (pending API integration)

April 4, 2026

• Added GET /recipe/{id} backend endpoint for fetching a single recipe by ID
• New repo function get_from_id() and Pydantic schema IdResponse (title, ingredients, directions)
• Built out the recipe/[id].tsx detail screen with collapsible sections: EquipmentOption, IngredientsOption, InstructionOption

April 5, 2026

• Refactored frontend — extracted shared Recipe interface into src/types/index.ts
• Created src/services/api.ts with axios HTTP client pointing to ngrok tunnel
• Two API functions: getRecipe(ingredients[]) → POST /recipe and getById(id) → GET /recipe/{id}