I use Gemini, which supports PDF File uploads, combined with structured outputs to generate Course Sections, Levels & Question JSON.
When you upload a PDF, it first gets uploaded to a S3 Database directly from the Browser, which then sends the Filename and other data to the Server. The Server then downloads that Document from the S3 and sends it to Gemini, which then streams JSON back to the Browser. After that, the PDF is permanently deleted from the S3.
Data Privacy wise, I wouldn't upload anything sensitive since idk what Google does with PDFs uploaded to Gemini.
The Prompts are in English, so the output language is English as well.
However, I actually only tested it with German Lecture PDFs myself.
So, yes, it probably works with any language that Gemini supports.
Here is the Source Code for the core function for this feature:
js
export async function createLevelFromDocument(
{ docName, apiKey, numLevels, courseSectionTitle, courseSectionDescription }:
{ docName: string, apiKey: string, numLevels: number, courseSectionTitle: string, courseSectionDescription: string })
{
const hasCourseSection = courseSectionTitle.length > 0 && courseSectionDescription.length > 0;
// Step 1: Download the PDF and get a buffer from it
const blob = await downloadObject({ filename: docName, path: "/", bucketName: "documents" });
const arrayBuffer = await blob.arrayBuffer();
// Step 2: call the model and pass the PDF
//const openai = createOpenAI({ apiKey: apiKey });
const gooogle = createGoogleGenerativeAI({ apiKey: apiKey });
const courseSectionsPrompt = createLevelPrompt({ hasCourseSection, title: courseSectionTitle, description: courseSectionDescription });
const isPDF = docName.endsWith(".pdf");
const content: UserContent = [];
if(isPDF) {
content.push(pdfUserMessage(numLevels, courseSectionsPrompt) as any);
content.push(pdfAttatchment(arrayBuffer) as any);
} else {
const html = await blob.text();
content.push(htmlUserMessage(numLevels, courseSectionsPrompt, html) as any);
}
const result = await streamObject({
model: gooogle("gemini-1.5-flash"),
schema: multipleLevelSchema,
messages: [
{
role: "user",
content: content
}
]
})
return result;
}
The UI mostly works offline once loaded in due to aggressive caching.
Downloading Course Content was on the initial Roadmap but I removed it since I wasn't sure if anyone would like the feature.
Syncing stuff is a real pain in the ass but I'll implement it if at least a couple people want it.
Thanks for the suggestion, I’ll definitely try to make the app as language inclusive as possible!
Also, sorry if I might’ve been too vague with the post title.
The app is just similar to Duolingo in terms of structure and the idea, however it’s not specific to language learning but supposed to cater to any subject, really.
For example, I personally use it to study for my university subjects.
Sure, 90% of you guys over there are racists and have a huge racism problem but don't go apply your racist shit to everyone internationally.
I'll continue having Xi an Pooh synonymous because I'm not racist and can actually do things in life without trying to make it about ""race"".
P.S.
God, where did Americans even get the idea that Asians look yellow?? Most of my family is Asian and is black. I know Asians who are paler than me (German).
It's like I'd color my face blue and someone calls me a racist, discriminating penguins.
To build an IOS app you need an Apple Developer Account, which is a subscription service for 100 bucks a year.
It's similar but not the same for Android. With Android you don't need an Account to build the App, just one to publish it on the Play Store - which also only costs 25$ one-time purchase.
Also, I can understand the financing on for-profit Apps like Sync. Shit costs so much money the dude breaks profit with just one person paying premium.
For hobbiest it's on the expensive side. I mean it's probably the most expensive part of the pipeline for most hobby Devs for a way overpriced service. But I get it if you really want to publish your Lemmy App on the IOS store and have the money to do it, just wondering if these people have any plans of breaking at least even via donations or such.
Thanks :)