Gemma 4 12B: proves de codi local, visió i àudio

This bottle is insane. Insane. Look at this. Got this drum beat. What to say? Today we're going to be looking at the brand new Google Gemma 412B. Now, this is a very, very interesting model for a number of different reasons, and I had no idea this was coming. So, one, that makes me mad cuz makes me feel like less of an influencer, but two, it makes me excited because this is a very, very interesting combination of a few very cool Gemma 4 models. Namely, it's basically like the E2B and E4B were kind of morphed with a smaller version of the Gemma 431B because this is a dense model, but it is natively able to understand audio and image input. And when I say natively, as we'll see later, I really do mean kind of like natively. So before we get into it, please do feel free to subscribe so I can get that 100K plaque. And additionally to that, I have nothing additional to say. So let's take a peek at this model, some of the cool things about it, and then we'll get to testing it. I do want to touch upon some key points that are outlined here in the release post for this model. I want to start with option or not option but number three right here because this is something I know a lot of folks are going to be excited about where they mention that this is a developer friendly size or small enough to run locally on a 16 gig of VRAM laptop or unified memory. Additionally, they did also release an MTP model which will speed up the generation speeds for this model. Now something else interesting is they have also released a new Mac OS desktop experience. I am using a Mac right now. So, I'm going to also play with this model through that because it does seem like an easier way to experience some of the audio and vision capabilities that are so cool about this model. Following that is where things get a little more dense. Get it? Cuz it's a dense model. No. Um, subscribe. But because they start to talk about the architecture and how exactly this works being like an encoder-free model. So, if we scroll down a bit more, fortunately we get an in-depth look. If we click on the hyperlink right here, which just brings us to this dedicated visual guide to Gemma 412B and how it architecturally differs and also works compared to the previous models which had encoders for text, uh, excuse me, for vision and audio. Now, there's a lot of interesting detail in this article. There's a bunch more images, but that doesn't necessarily mean it's going to be simpler to understand from a conceptual standpoint. So, I don't like reading things verbosely back to the viewer, but I think I'd like to skim a few choice pieces of word from this article to make

a little more sense. So, to give an LLM the capability to understand modalities other than text like audio and vision, much of the processing is generally not handled by the LLM. Instead, a separate encoder driven by attention is used to first process the input into embeddings. This tends to be a small transformer model and uses attention to process the input much like an LLM would. So if we go back to the chart that is listed on the page right here, we see there was mention of that being a small transformer model which would be attributed to the numbers and sizes we see right here where these are basically small additional models that help the processing of these different forms of media or audio and video or image if you will. So this model the 12B having basically nothing for those ties back into what is being referred to right here. So the resulting embeddings from that process cannot be used directly by the LLM. They're typically different dimensions and might behave differently in any dimensional space. A common technique is to use something called a connector. It converts the output of the encoder which we saw previously in these other models. So basically right here and here. So the connector converts the output of the encoder to the same dimensions as the token embeddings. So basically to summarize and don't quote me specifically on any of this but the old model and I have put this in excali draw here because I found this specific metaphor to be somewhat understandable and more or less fitting. So the old model is like a chef getting ingredients prepped by specialist assistants which is kind of what we saw in that chart with the additional encoders that kind of fed the things to the model. It was like getting those ingredients prepped by assistant chefs but that adds some overhead and adds more folks in the kitchen and things of the sort. The encoder free model or GEMA 412B is a very capable chef getting roughly chopped ingredients already delivered to them and they have the training to prep, understand and cook these dishes without needing the additional inline chefs to prepare those things for it. So that's kind of a very very rough summary of what's being talked about in that article, but for the purpose of a simple first look and test video, I think it's probably decent. And for anyone more interested in understanding this stuff, honestly paste the link or the contents of the article into your favorite somewhat competent LLM and then ask it to explain it to you in a way that you understand. So say give me metaphors pertaining to some hobby you have or something of the sort and then things will start to make a lot more sense with that. Now I think the next thing to do is start testing this model. Now, first and foremost, I do have this open in LM

Studio. And for today's testing, we're going to be using this on an Apple Mac Studio M3 Ultra 256 gig. Perhaps a bit overkill for this specific model, but because they did release the Mac app as well, I want to play with that. So, we're going to use this system for the test. Additionally to that, I do also want to mention a couple of things. So, if we go into the quantizations for this model, just from Hugging Face right here, we're going to see the Unsloth one. And while I am using the GGML 8bit quantization for today's testing, I want to specifically direct our attention to this Unsloth blog post right here because not only does this have the correct recommended settings for the sampling parameters for this model, but it also has instructions on thinking mode and enabling thinking and things of that sort. So when we do play with this model in LM Studio, we're going to see that it is showing us its chain of thought and the instructions are there in the Unsloth blog. So with that, let's start with our tried andrue browser OS test. Now this model sits at an interesting cross-section because I'm probably going to be giving it a lot of the coding related tasks that I would give to state-of-the-art Frontier big models because it is a 12 billion parameter dense model and I really want to push the limits of its coding capability, at least in terms of our initial browser OS test. So I do want this to include a 3D GTA clone in the result. So this may be one of the shortest results I've seen for this test. Keep that in mind when we click on it. Okay, it was under 400 lines of code, right around 390. Let me close all these other tabs here since we're done with these. First and foremost, do we have a right click? Okay, we don't. But again, it's not specifically denoted in the prompt. Let's click on our start menu as denoted by this rocket ship. Okay. All right. So, not necessarily 100%. on the ball. I would have expected a little more capability here, at least in terms of some of the UI elements appearing or working. Maybe not the full-on GTA clone. Okay, seems like we have some simpler syntax errors. I'm going to yell at this model. The user is angry. Complete embarrassment beneath your oh so great weights. They pointed out specific console errors, several reference errors, and an unsafe attempt to load URL file, which we can ignore. I apologize for the oversight. The syntax error effectively killed the entire script execution, which is why all subsequent functions were undefined. All right, let's see if it properly fixed this. Very good. We're not getting any errors. Now, we still don't have a right click, but that's okay. Let's check our start menu. Very, very, very good. The special feature, I saw it talking about

like delta time. I'm not quite sure what's up with that, but we're probably going to wait. We'll try that last. Here's our notepad. Okay. And again, I have to consistently remind myself that this is a local 12 billion parameter model at a Q8 quant because a lot of the tests recently have been much larger models. So I need to reframe my expectations for a 12B. This is so far actually a decent result. Calc. All right. 47* [snorts] 52. That was just a stupid math thing to type. 2000. What's 7* 52? That's easy. 364. So 2364 plus 80. Very good. Okay. Uh system info. Good. CPU load. Memory 16 gigs allocated. Warp status normal. Okay. We'll go back to that warp status. Now the big one. Does our micro GTA result work. I didn't really expect it to to be honest with you. And I'm not going to negle. And we can try maybe a dedicated game result. [snorts] H. It did spawn a specific error though. Ah, this might actually be something that's worthy of having it attempt to fix because it's not horribly difficult. I'm giving it a chance to fix that. Void runner. Void runner. More like I can't say that. But so this is actually a functional 3D game. Arrows to dodge. Now the issue this might be a good time to try this feature that messes with the delta time of the operating system, the special feature. And now that makes a lot more sense. And that's probably one of the best visual demonstrations of said feature is if we turn that special feature back off now everything is a disaster. If we turn it on significantly less so. So arrows to dodge. All right. Uh still no keyboard but the fact that it actually did this. This is it's not a stupid space shooter game which every other model minus like one or two has done. So I'm actually happy to see that. Change wallpaper. I don't like orange. All right. I think this will have actually fixed the micro GTA clone. It did. This model's getting insane in the title. No if ends or buts about it. This model is getting insane in the title. It's actually like the camera movement's good. It's fluid. Really, the only thing I'd need is like the James Web telescope to actually be able to see this thing. But aside from that, [laughter] this really is a actually so far. I like what I'm seeing a lot. This is going to get insane in the title. The driving movement here is fluid. This model is very good so far. I'm quite pleased. No right click though and no clock, but I'll let that slide. I'm going to go a little out of order here because I am deeply curious to see how this does with a 3D printer simulation test. So, we're going to run that now. We'll still run some of our other traditional tests, but this may be

a longer video than normal. Though, I am very, very excited to test this model. I should have also mentioned in the beginning that this is Apache 2.0. So if there are any comments that say like you didn't mention Apache 2.0, just know that they didn't watch the video thoroughly. All right, we have a couple syntax issues. So I'm just instructing it what to fix. I've also given it some specific like this is the specific line where you messed one thing up. It will fix that finally. Okay, so there were a few errors that occurred. I think this is probably the third or fourth time really. A lot of what it came down to was it just was messing up the import map and then there was one extra um bracket on one line that it fixed. So let's start our print now. You know what? For a model of this size, I am okay with this. I know that may sound weird, but it's doing it in the correct shape. It's a very artistic rendition of a 3D printer, but more or less we have the shape of the printer. We have a nozzle that is moving in the correct pattern it would for the specified shape and it is actually extruding. They're dots. This it's almost weird. Like if I zoomed in right here, it'd be like, whoa, what sort of test is he running? And then you zoom out and it's like, oh, interesting. It's an ultra sparse print. The movement though of the camera, like if I click and rotate like here. Okay, I'm clicking, spin, let go. It's got nice fluidity to the movement, which I noticed in the GTA sim as well. This model's smooth. All right. Uh that's only layer 16. Oh my. Okay. Uh let's try the circle. Oh yeah. Good. And it's just the nozzle's moving in the correct shape. This looks like the coliseum. [panting] Okay. Uh I think we can just use our imagination to see what the Oh, it started printing the triangle on the circle. Can I swap to the square now? and it will do that. Oh, what? This is kind of cool. Now, don't get me wrong, that's not supposed to happen, but I'm glad it did because this is [laughter] nonetheless. Very cool. Next up, I want to give it a multimodal test where I've given it this AI generated photo and instructed it to create an SVG graphic that is a faithful replication of this photo. All right, let's check our image.svg. [laughter] You know what? I'm going to zoom out. Yeah. All right. Well, [laughter] the resemblance does exist in terms of the color palettes it's chosen to use, the relative orientation of the folks in the photo and the expressions. You know, it showed good ability to capture uh color palette and maybe composition of said image if not one to one. So, I'm giving it a test to create an accurate

replica of Jerry's apartment from Seinfeld. I've also included this reference photo. It should do this in 3JS. I want to be upfront about this that this is totally totally out of scope and beyond the capabilities of this model to do this. This would basically be like trying to take a very expensive sports car like on an off-road trail designed for Jeeps. Even maybe more out of scope than that. We're going to just see what we can get. I cannot generate and host a complete 3D file containing every detail. However, I can provide you with the professional architecture, the precise scene coordinates, and 3JS boilerplate code required to build this. I'm okay with whatever it gives me here. I just want to see this is almost just a test as well as to see how it goes about doing things. And it's good to see it does seem to have an assessment of its own limitations here and saying I'm not going to be able to do this, but I can give you something to get started. So, here. All right. So, I fixed this manually. It was just again like kind of an import mapping issue which it's had in all of its 3D things, but as a scene, as a barebones scene, it did kind of what we asked for. It's a very, very basic apartment square, and it told us it's not going to be able to do the full thing, but it can get us started, and it did. And again, I'm noticing the fluidity to the movement is still pretty darn good. So, I'm impressed so far. For a tiny little model to accurately make a decently made space like this with some basic assets in it and some different colors is cool. Next up, we're going to try the beautiful static subway scene. I've also specifically told it to use an import map that will avoid path resolution errors, which is what's been happening in all of the previous 3D results. So, it did this in two different scripts. So, I just put them in the same directory. And I'm now wondering if I had named one of them incorrectly. Okay. So, this needs a bit of manual handholding in terms of its coding and things like that, but again, we received a semifunctional result here of a simple subway station. I don't know that the brightness slider really did anything. But again, this is a small little map that would be probably very very very decent for some form of FPS. Uh so uh we will come back to this, but I do want to now go and basically try some of the native capabilities in this new Mac OS app that they have released as well. So the audiocribe or ask image portions of this app don't seem to allow us to select the new 12b model. It does allow us to select it in the AI chat as well

as the agent skills pane. Again, it's very possible that I may have done something a miss here. So, it seems like the only way I can actually Yo, what's up, bro? How you doing? Oh, yo, what's up, bro? How you doing? Okay, you have to Yeah. All right. That's That's just not uh All right. Hold on. Yo, what's up, bro? How you doing? And then we can just send that and it will understand what we had said to it. We see this is directly just an audio waveform. It's not even showing us a transcription of what we sent it. So, this kind of ties into its native, yo, I'm doing great. Thanks for asking. How's everything going with you? What's on your mind? Can I not send a follow-up audio message? I would imagine you can. It's just probably roleplay as Steve the PC repair man. Now it's hard to really I mean we're getting one demonstration of it. So being able to showcase the voice input and the image input for this is not as smooth as I would have thought it would have been in this specific app. So we're probably going to course correct a bit. However, it does have this specific Asian skills section where if we click on skills right here, we can see there are 10 specific built-in skills. Some of these are kind of interesting. There is one right here, kitchen adventures, where it ask acts as a dungeon master for a textbased adventure set in a world where everyone is a sentient kitchen appliance. Additionally, it just has some like Python capability as we see right here. It can write, package manage, and run custom Python scripts using UV to solve complex tasks, run data analysis, or generate visual plots. So, unfortunately, we see it encountered an issue because the UV is not installed on this system. I'm not going to do anything to rectify that currently. I just wanted to get a feel for this application. It does have some interesting things to showcase and definitely for folks interested in running this on a local machine. The download size for the specific version of the model that is being used here was like 7 gigs. So, you probably could get away easily with running this on some 16 gig of RAM unified system or a VRAM system with like 16 gigs. This would be good for playing with this model and testing it. However, now after I've looked at this, I think for the purpose of our testing, we're definitely better off just staying in LM Studio. Next up, I'd like to try some multimodal based things. So, first we're going to give it a photo reference that was AI generated of a website UI mockup and instruct it to replicate that website. So, here's our replication of the site. This is actually a good front end compared to what I expected. Okay, give me a second here. So, we can see things

kind of start to fall apart when we scroll down. What I kind of want to say is this does not look like an old school website front end or something one would expect from a local model like a Quen 3 or a Quen 2.5. This looks like a more modern UI take, at least in terms of what I've seen from AI models. And that's actually kind of exciting. So, it's somewhat faithfully replicated portions of the site, such as the business names right here, which are all shown nicely. No errors. So, it just was a simple take on that. But again, that's kind of a difficult image to give it. I'm asking it to expand this with a graph UI that is functional as well as testimonials and pricing because I am impressed with the front end just from what I saw in this one test. So, here's our updated front end. Okay, good. We do have a nice little interactive graph or at least something that looks a little better trusted by engineering team. So, it did add in the customer testimonials as well as the pricing cards. I'm going to say for a 12 billion parameter model, this is not a bad front end at all. I'm actually quite pleased and we have to remember this was just based off of an image reference as well. So, this definitely warrants some additional front-end capability exploration in my opinion. All right, I've put this in open code now and I'm going to give it the C++ skate game test. This is very difficult for a model of this size. I don't even know what's going to happen, but we'll at least be able to witness some of its agentic functionality. So, it is starting in plan mode, of course, and we see the thinking is enabled, and it's just outputting what it thinks it's going to do. All right, I'm going to actually allow it to use RayB. So, it asked some clarifying questions. I've just swapped it into build mode, told that it can use Ray Lib, and it just needs to run on this specific operating system. So, build it. I don't know 100% if this is going to work, but we'll see what we get. Okay, that was very quick. Now I want it to go ahead and test the compilation itself. Okay, good. So it's noticing that there is no ray and it's saying you basically need to install it through homebrew. Well, that's what it will end up saying. Okay, I will do it. It's not going to install it for me, which is fine. So I'm just saying done because I wanted to see if it understands. Okay, he installed Ray Lib and now still cannot find it. Usually means the include path isn't being automatically picked. I will try to locate the header file first. Is it still? No, I think it stopped. Okay, I'm going to just give it the output of

brew. Again, this is difficult stuff for a model of this size, especially working in like a a gentic harness like this, but I find it very interesting just to see even if we don't get a functional game here. I think it's cool to see this thing orchestrating and handling agentic functions. Compiler is failing because it's defaulting to an older standard that doesn't support. Okay, I'll rerun the compilation with the correct So it attempted a bunch of times to just replace the specific line that was causing problems. It didn't work unfortunately. So it decided it was just going to rewrite the entire thing. So it worked its way around that issue it was facing. And now it's saying okay the rewrite operation introduced some syntax errors. Now I'm going to fix these. Now as we see right here in lie of just editing those specific lines, it's just rewriting the file each time. um which is not necessarily the most optimal, but it's very interesting just to see how this goes about functioning in here. This model is insane. Insane. Look at this. Please don't I am I am dumbfounded. Okay, I am dumbfounded that this [laughter] that this worked because it was making errors. it couldn't figure out how to edit like the single lines or it was messing up with that and then it just decided I'm just going to rewrite the script each time and and it did. Look at this. This model's a freak in a good way. I'm Wow. Yes, we allowed it to use Ray Lib, but still it's a 12 billion parameter. This thing really, this will democratize intelligence to those who do not have like mega computers at home, which is awesome. This is awesome. This is really exciting. I know that probably sounded sarcastic. It's not. Is there a right click? Nope. Fail. No, I'm kidding. Okay. Well, that was that exceeded expectations to such a high degree. I'm giving it something really simple. I've just sent it this photo of a very, very barren wireframe and said, "Turn this into a beautiful high-end website for Steves." I didn't really tell it what Steves was. I don't know what Steves is myself, but we'll see how well it adheres to the header, subheader, three cards, and all of these elements here. We need to move away from boxes lines and towards sensory storytelling. Okay, since you didn't specify what Steve sells, I'm designing this with a minimalist luxury aesthetic. I think Hermes meets Apple which did happen because they have an Apple watch that's that if you're one of those people who likes things like that which I'm not here's the blueprint to transform the sketch into a worldclass website. All right so I'm telling it it's a high-end watch brand in lie of placeholder images just make SVGs that look like high-end watches since I can't generate actual it'll figure it out. All right, here's our Steve's watch website.

Okay, the pinnacle of herology. Interesting. This could be seen as like luxurious just based off of like the style and things. It is the SVG watch background that is perhaps letting it down a bit, but still the heritage, the Kronos, the deep craftsmanship defined by the silent passage of time. Steves, London, Geneva, New York. This is This has the bones of being like a high-end site. It's just the lack of like real assets that lets it down. But think of it, if this had like a cool image right here and then some watch photos and then maybe a photo of Steve, this would look really good. Well, for the model, considering the size. Next up, we're giving it the Flight Combat Sim test. It should be able to knock this out pretty well. So, I actually want to showcase something. The Flight Combat Simulator result kept having a bunch of small annoying issues that were preventing it from working. Things like what we see right here. So, I actually pasted this script into chat GPT. I said, "Do not change anything. just fixed the double like parenthesis as well as any potential import errors. So, I want you to see right here everything that was fixed with this. And now we're going to take a look at the fixed result that chat GPT helped do. But keep in mind that this was created by Gemini Gemma 4 12B. So, the whole point that I'm trying to highlight here is the model itself did most of this. It had a bunch of small like annoying errors. But as models become more capable but also more expensive, we're going to start seeing a lot more the concept of like routing different tasks to different models. So say I had chat GPT create this, which would have been expensive in the scheme of things versus just doing this locally on my system, but we had Gemma 412b do it and then we had chat GPT clean up like a couple of bug fixes in a smaller pass. It is going to be likely that a lot of folks in the future will use different models. A small cheaper model for kind of like the scaffolding and what we see right here. Look at that. We even do actually have the enemies there. And that's very cool. And then a more competent larger model perhaps for some bug fixes or even just like fixing them oneself and you know cuz that's also I guess a possibility. But the whole point here is like this model it's made a lot of simple mistakes in terms of like basically it's 3JS code creation which is a pretty niche thing but it's the foundation is fantastic and this wasn't done through open code or anything where we could have also just wired it into open code and said okay figure out a way to just check the error logs that you're getting and then it

absolutely 100% would have been able to autonomously fix these issues. I just kind of wanted to speed things up but also showcase that like chat GPT or different smarter model with a smaller cheaper but still capable model pipeline I guess could be said. So this is exciting in my opinion. Now in the meantime I had Open Code take a stab at the subway scene that we had received and for 10 minutes it was working on turning that into a very cool FPS. I don't know if it's going to work or not. Okay, so unfortunately we did not get anything, but this will be easier to have it fixed because it is working in open code already. Oh, this may be entirely my own fault. All right, so there is one issue from our Subway FPS improvement. I might manually help it here. Okay, so my tip seemed to have pointed it towards the correct fix. So we're now just instructing it to fix it. Something I noticed sometimes in open code is it will kind of pause and you need to tell it like okay build it good it found some additional issues as well so it's changing those it did it it's somewhat functional somewhat not functional but it did turn it into a very simple FPS using that map just from within open code now okay the speaker is on. Oh, the speaker's not even hooked up. This is hooked up to a big big sound system. We would have heard it if it was working. We do have some muzzle flashes. We do have a very simple weapon model as well as some enemies here. I'm not really seeing any health logic or things of the sort, but the whole point is it did actually kind of improve this initial result into an FPS. And this was all just done through open code. needed a slight bit of handholding for some simple syntax issues and things, but still, I mean, for the size of this model too, I keep having to remind myself like it's not a big model at all. This is very, very, very impressive. So, the final thing we're going to do is a drum kit simulation test. I have changed this prompt just a slight bit because I don't want to deal with like 3D asset issues when we want to just see how the sound works and everything. So, I've told it it should just be 2D, and I removed any suggested like use 3JS or tone.js. I just want to try to have it do it as simply as it can. All right. So, here is our drum kit. That slaps. And I told it to do 2D, so that's not an issue. I can I All right. Toggle key map. All right. Let's try our auto play. >> [laughter] >> Okay. I mean, technically, yeah. Uh, track two. Okay. Ah. Model testing every day. Got this drum beat. What to say? No. All

right. That's it. I'm done. That's going to conclude this test. Overall, I probably spent a longer time than I had anticipated testing this model. And I do have to be honest with you, I didn't really focus too much on like the native multimodal capabilities because uh this thing's awesome at coding and that totally usurped usurped all of my intentions of checking any of that other stuff. We did some multimodal coding tests. We saw just quickly the voice transcription where it understood that just by playing with the application that was released for Mac as well. But honestly, the capability of this to pop out code for tests that we run with Frontier models was absolutely fantastic. Sometimes it makes simple syntax errors and things, which is why I wanted to highlight having something like chat GPT fix smaller errors when this model builds the foundation because that opens up a ton of possibility. But really, this model is very, very impressive. using it through open code even and having it create a C++ self-contained skate game was just not something that I had on my bingo card for testing Gemma 412B. This model really really really really is very good. The skateboard games right here. So, we didn't even open this. We didn't put it in its own directory cuz I didn't really have any like I didn't think it would do this. It took a little bit and I had to install some brew packages but it told me exactly what to do. It found some issues. It fixed them. I mean, that just totally totally blew me away, truthfully. So, really, I'm not going to spend too much time on like a results overview or closing thoughts section. I think it's very clear just throughout the duration of this video how impressed I am with this model. This really, I would believe when factoring in being able to be run locally with raw capability, I would say this is perhaps the best local coding model that currently exists because it's really, really good [laughter] and it's fairly small. So again, I tested it in 8 8 bit quantization. I don't know how these capabilities would scale in a Q4, but there's definitely raw capability and intelligence here to a high degree. I'm very impressed. It seems also pretty good at front end. We didn't explore that a ton, but that is definitely something that is worthy of seeing more. So that's going to conclude the first look and test of Gemma 412B. a very very very impressive model that I think a lot of folks are going to be extremely pleased with. So, thanks for watching.