Muse Wonder a prova: què mostra el test pràctic de Bijan Bowen?

Look, someone else is skating. [snorts]

Get out of here. This is how dogs feel

when they start barking, when they see

other dogs out the window. I understand.

Meta has come out with their Fable

competitor, which I think now would be

an Opus competitor cuz I think Opus

beats Fable in a lot of things. However,

this is Meta's Muse Wonder. Also camed,

I believe, Watermelon based off of some

of the teasing that was being posted on

X with photos of Watermelon and then

references to this model. This is

supposedly just based off of what we're

seeing right here, a true and proper

Frontier state-of-the-art model. Now,

I'm extremely interested in testing this

because in some of our previous testing

of models like Muse Spark 1.3, it did

not necessarily seem like the benchmarks

reflected its capability. At least in

our suite of tests, it was actually the

sole defining point in which I decided

Deepsw S. 1.1 was no longer necessarily

um something I could trust properly for

coding capability. So, let's see how

this model really is. Now, I do have to

say, of course, we're going to go over

this a little bit more in depth. Don't

forget to subscribe so we can hit that

80k number and then crawl up on to the

100K. And additionally to that, I

subscribed. So, as we're going to see,

there are two variants of this model

that have been dropped. One being

slightly less powerful and one being

slightly more powerful. In order to

access the more powerful of the two,

which is this Muse Wonderful, you need

to subscribe to the Muse Code plan. So,

I just subscribed to the $50 a month

plan, and I have two simultaneous

sessions going right here. The browser

OS test v2.7

and then the C++ skate game. This is all

I've done. This model's been out for

like 25 minutes and it's already used

52% of my 5h hour usage on that $50 a

month plan. and my weekly usage is

already at 17. So, the two models that

have been released are Muse Wonder,

which is the slightly less capable

variant of this, and then Muse Wonder

Full, which is the highly capable

variant of this, slightly more capable.

They start out here by just showcasing

the leaps in capability that have

happened over the recent releases. And

Meta really has been doing a lot of

impressive work lately from Musepark to

Musepark 1.1, 1.2, 2 1.3 and now we have

Muse Wonder right here. We can see the

leap between this and what I would

assume to be its predecessor being that

it is um the leap before it. Here this

is a pretty significant jump in

capability. And then even for the full

version of Muse Wonder, an even larger

one only second to Opus 55. We have

Claude Fable 51 here as well as Argon

which is still currently not yet

released. So folks may I mean you can

only judge this based off of this chart

right here. No one's actually really

used it yet. Now, of course, in this

blog post, we see a lot of the

traditional things we're used to seeing,

such as the benchmark charts here. And I

do like that both of these are shown

here benchmarked as well. Both run on

the highest potential reasoning effort

for these models. And they really do

seem to be performing very, very well. I

mean, this is, I think, the highest deep

SWE score that we've seen. Now, they

mentioned two new things right here,

which I've not really seen mentioned as

often in model releases and things like

this. the concept of active memory and

active focus. And I think what this ties

back into is if we scroll all the way to

the top right here, they mention this is

soon going to power experiences in Muse.

Now, Meta has been going very heavily in

the personal agent space. I think a lot

of folks very much like the agent. So,

this is something that seems to have

intertwined capabilities, empowering

better, more capable agents that know

more about you and things like this. I

do have to wonder though if the usage

that we're seeing right here, I mean,

that's ridiculous. I don't even think

we're going to get a complete result out

of this just from a browser OS. And this

has only been working for 14 minutes and

we're going to hit our usage limits. So,

off the bat, I am kind of ticked off and

that seems like a waste of $50. So, the

pricing for Muse Wonder, which is the

slightly less capable one that we had

seen in those benchmark charts. It is $4

per million in and $20 per million out.

This is expensive. And what is the

comparable pricing for this? I feel like

it's Opus 55 price.

Yes. So, this is exactly the same price

as Claude Opus 55. Additionally, what is

the comparable GPT model? I don't think

there is a GPT model currently at this

price. Yeah, so there's no comparable

GPT model for this price. So, GPT61 Soul

is half the price, at least for Million

Output tokens. And then if you would

like to use the full version of this

model, you can only do so currently

through the Muse code subscription,

which you can see [laughter]

it does seem like we have two results

now to take a peek at though. So let's

just jump in and take a look at our

browser OS. I am really expecting to see

fantastic high-end results here,

especially considering that it's

benchmarking that way and it also uses a

lot of usage. Okay, we have Nova OS. All

right, we do have some movement. A

relatively

dense bit of welcome,

but not bad. So, this is probably like a

$25 browser OS

that I like. These are our wallpapers,

the dynamic ones. Probably stick with

that actually. No, because that's giving

me like optical illusion like vibes.

Maybe we'll stick with this. Let's check

the big one.

All right. There's no right click. No

command center. So, this is our special

feature.

Okay. Good. I can speak to it at least.

Allow visiting.

Give me a refund.

Oh, okay.

Refund Muse code subscription now.

Okay. So, it's, you know, why are you

special? I'm Nova

wallpaper plasma.

All right, let's just check our

Okay. Okay,

not bad, but not opus 55 level. I know

this is one specific test, but you know,

it's not Opus 55 level. It's not Sonnet

55 level. It is around haiku 55 level.

All right, crash. Watch your health. I

think we're stuck in like a mesh

collider corner there.

Can we steal one of the police vehicles?

No.

We lost the police. I don't know how,

but I'll take it. All right. Overall,

let's keep going. Next up, we have our

Starfall Defender 3D. Okay, the

traditional.

This one's actually It's not bad. I do

like the effects when these asteroids

blow up. That's actually kind of cool.

It's competent enough. Okay, ship

destroyed. Next up, we have our mail

application. Acceptable. It's got meta

references to

apps on here. I'm still just stuck on

the fact that our usage is burnt out

totally. Notes and files. Notes and

files. Is this a unified?

Can we save this as a txt? Good. Okay. I

have not actually seen that in a bit of

time, so that's good to see. terminal.

All right. There may be a redeeming

application down there. 76*

6

472. Uhoh.

Paint. That was there already. I didn't

do that. Is this like a muse looking?

It's like their logo, which is

interesting.

Cool. And then we can also set our

custom picture as a [laughter]

desktop background. Let's change that

back though too. Okay, good.

Settings and wallpaper studio. Okay, I

do like the We have our painting there

as well. Particle nebula. And we can set

the animation speed as well. Good. I

like the star warp one. And if it's

slower, it's a little No, it's still

kind of like weird. Dusk waves accent

color. Okay. And it changes some

elements on here. Now, something I'm

quite eager to see. Okay. Well, less so

now after seeing it. Generative synth.

Calm out. It's designed to calm you down

after it blows through your usage

limits.

And then finally, we have welcome. And

our special feature was just this one

that we could speak to and like so

open GTA

good. Overall, I'm going to refrain from

judging anything too heavily based on a

first result and we'll move on to the

other result we did manage to get which

was the self-contained C++ skate game.

So here, okay, good. This is definitely

better than I had

worried.

This is actually there are elements here

that I'm pleased with such as the fact

that the players actually pumping. We

can interact with pedestrians. There is

some, you know, immersion here and the

board moves independently from the

player. Now, unfortunately, this is

still like not even in the same galaxy

as the GPT. This is GPT6 Soul. Um,

actually 61 Soul did a significantly

better job than this. Good. And we do

even have a balance feature. And I will

say the human model for the skater is

not really that bad. We do have a puddle

and we have a fountain. Let's see if we

have the typical fire hydrant spraying

water onto the street. Oh, okay. And

there's also like

a a road. And

that was actually kind of a sick trick.

However, I I can't help but be honest

that these results are completely

tainted by the fact that this was

two of these chewed up, forget the 5h

hour window. Two of these chewed up 36%

of a weekly limit on a $50 a month

subscription. That's ridiculous. That's

just That's not That's crap. That is

crap.

All right, so I'm going to upgrade this

plan. Currently, we're on the $50 a

month one, which gives you 20 times more

usage than the everyday usage plan. This

gives us 80 times. So, we should get

four times as much usage. I'm We're

basically going to just test whatever we

can till this usage runs out and then

I'm probably done with this video.

All right. So, on X High, we'll be doing

the subway FPS test. This is the one

where the train comes when we clear a

wave and then we take the train to the

next station. And it's on X high here.

It is not on max. We'll keep an eye on

our usage. All right. I don't know that

the subway FPS is fully done, but it's

been like play testing it for quite a

while. And because we really do need to

be somewhat cautious of the usage, we

need to kind of

like um you know, be careful. All right,

I'm going to say this is better so far

than what I noticed with some of our

previous SK game results. Let's check

out how the enemies look. Okay, they are

a bit simple.

Let's just Oh, wow. All right. That's

how I feel having paid $200 for this

subscription. All right, there are no

bullet holes. However, this is the one

where the train needs to come get us.

So, all right, let's take a look.

Good. Not bad. Not bad. Well, not bad.

All right, mind the gap. I'm not

actually going to give it flack for that

issue where like the stripe of the train

creepy. I like that where the stripe of

the train covers the doors because that

I've seen with a lot of models. I think

some of the clawed ones and stuff did

that as well. So, all right, let's turn

this volume up and let's just get ready.

Oh, okay. Don't ask me what that is, by

the way, cuz I don't know. Okay, the

clock is

Was that like a cool an ammo pack? We'll

we'll go to one more station and we'll

see if it makes the enemies larger when

we get there. Good train shake is a

sentence I ever thought I'd speak.

All right.

Stacks of papers. Not bad. Okay.

Too easy.

Oh, there's a big one. Okay, good. So

that's just sometimes what I like to see

is

and this one's tougher. All right, let's

see what happens if we lose. Okay, end

of line. Overall, it was

not horrible.

Pretty simple, but again, I kind of am

just more focused on preserving usage.

Normally, I'd like to be like, okay,

like decent start, make it way better.

Maybe we will for this one, but I also

wanted to do that for the skate game as

well. So, we may come back to some of

these.

>> Hey, Perry, could you turn your volume

up all the way?

>> Okay, I'll raise it to the maximum for

you.

>> So, this is Perry. Now, you may be

thinking, Bjan, what are you trying to

sell me? Nothing. Because I do want to

point this out. This is a robot I made a

few years ago, and we recently had some

clawed model overhaul the UI in the

software. This is on GitHub if anyone

wants to build it. It is 3D printable.

All the parts list is there and

basically you can print it in whatever

color you want and then speak with it.

Currently it uses the GPT real-time

voice AI but you can fork it to work

with local or whatever you want to do.

So I just wanted to showcase this as I

think it's pretty cool. What do you

think Perry?

>> That sounded great. It's

friendly.

It sounded great. Clear, friendly, and

confident. There's a nice come hang out

with this vibe. If you want to tighten

it, you can try.

>> No, I don't want to tighten it. Thanks.

Okay, so back to the test, but the link

for this will be in the description of

the video if anyone wants to build this.

Next up, I want to try the non full

model. So, this is Muse Wonder 1.0.

We're running it on the maximum

potential effort, and this is the one

that is available just like through API

and stuff like that. It scored a little

lower than the full version on

benchmarks and stuff, but I am

interested in seeing how this does

because you never know which one may be

better for certain tasks and practice

hopefully. This is the Blender and GDAU

backyard pool party game test where it

needs to create a backyard pool party

where you jump off a diving board in

Blender and GDAU. One just a little

newer that we've run in some recent

models with some varying levels of

success. All right, so in 9 and 1/2

minutes, we've received our pool party

result using Blender and GDAU. Again,

this is just the Muse Wonder 1.0. This

is not the full version, but they were

pretty close on benchmarks for better,

for worse. So, with that, let's now take

a look. Okay, decent little SVG icon.

[snorts]

Good. Um, okay.

It's just so simple in terms of like the

maybe we have to click.

Okay. Big Bubba, Luna, Captain Cannon,

or Kid Rocket. I think we're going to

pick Big Bubba. Okay. Um,

this is

I'm actually not as infuriated with this

as you may think.

It's got a decent aesthetic to it. I

just wish it was a little more like

shift is to spread eagle. Okay. Oh, and

we can actually

Okay, [snorts]

let's just [music]

we can move as well.

[laughter] Oh no. Okay.

I [snorts] mean, there are elements like

what we've wanted, like the dog running

around, the bounce and things like this.

I'm wondering if this is a model where

you just not every model's a oneshot

stunner. So, it's possible that we need

to push it a little more. Let's just

see. Okay. So, all right. So, I'm

telling it, give me like a full visual

overhaul, AAA quality, which, you know,

we'll see what we get from this. It does

seem pretty fast, though. And this is

our first test with the just the Muse

Wonder one, not the full version. All

right. In just over 16 minutes, we

received our updated version for the

backyard pool party game. [music] Okay.

The annual Suburban Splash Off.

Okay.

I must admit I'm having a bit of trouble

[music]

differentiating some of the

[snorts and gasps]

improvements so far. Okay, we get an

instant replay now. Not bad. Um, I think

it made some of the background NPCs

maybe a little more noticeable. Big

Bubba definitely has a stomach that

coincides with his name. Um,

oh,

okay.

There birds in the sky. The dog is

running around. [snorts]

A primitive dog. Oh, [gasps]

wait. [laughter]

You know what? Let's get creative.

Don't ask me where that came from.

But we'll see what we get. And this is

again not using the full model. This is

using the um non full model, the one

that's a little lower on benchmarks, but

perhaps less usage. All right. In about

10 minutes after just a random thought

came to mind, we turned our backyard

pool party into an like a beat him up.

Let's just see what happens. I think

we're going to go.

All right.

[crying]

Unfortunately, it's it's lagging pretty

bad, but definitely kind of

kind of what I had hoped to see. Just in

kind of a molding of two of our highly

regarded tests being Street Eat and then

Backyard Pool Party.

We won good. The big issue I had with

this one was just the lag.

We fell in the pool. That's actually

kind of cool. I like that it left that

in.

I'm just trying the special. I think

we're going to fall in the pool.

All right.

I don't know what to say about this.

Overall,

it works very quickly. It seems like the

Muse Wonder model, so not the full one,

but the one under it seems very token

efficient and fast. That was on its

highest potential reasoning effort. So

interesting, and I don't quite know what

to make of that. All right, cool. It

seems like the um usage reset

thanks to me. Next up, we did the robot

arm test where it needs to move the

little orange toy car from one side of

the mat to the other. This is purposely

designed to be tough because they only

get one static camera angle and depth

perception is basically not there at

all. So overall, I will say good things

are it moved cautiously. It was not

trying to rip itself off the table or

thrash around like we saw with the

clawed results which for some reason are

just like odd. They try to escape. It

did a nice job of just kind of slowly

trying to do things. But as we can see

right here, even if we just quickly

cycle through some of the images that it

itself took and was looking at, it

unfortunately never really understood

like, wait, this is a depth perception

issue. The vehicle's not getting moved.

Maybe I'll try to search around a

little. And ultimately, after about an

hour and 15 minutes, I opted to stop

this. Now, this was not the full

capability model. This was just the Muse

Wonder on max effort mode. I find that

sometimes like full-on power and a

higher reasoning effort does not

necessarily help this task. So overall,

it did not successfully complete it.

However, it didn't make any destructive

actions with the ARM. So for that, I

will say its cautious approach was at

least nice to see. All right. So, one of

the things that I wanted to go back to

and now that the usage issues were

identified as a bug, so we're actually

able to do this. The full version of

this model on Max, I basically gave it a

follow-up to the Skate Game saying this

is pretty poorly done. I'd like you to

redo it. Now, additionally to that, this

was just given a single static reference

photo. I believe this one right here.

And I told it to make it look more like

this and just make it overall way

better. This was started from an

entirely new session. So, I opened this

folder and I created a new session. So,

it didn't have the history of the actual

creation of this game. And I said, make

it look more like this. Now, we see a

ton of reference shots right here of how

this has been doing. And this is really

coming on to like 2 hours basically.

Let's take a look at this. Okay, look at

that. Look at the recording camera that

it's actually showing us in the bottom

right. I'm going to say this right here

is a significant significant significant

improvement and I am rather pleased with

what I'm seeing. Now, yes, we would have

liked to have maybe seen this. Maybe not

to this degree of insaness off the bat.

However, this shows a pretty promising

ability to improve itself based on a

reference photo right here. This is

really a very, very faithful replication

of the source image I gave it. Look,

someone else is skating. [snorts] Get

out of here.

This is how dogs feel when they start

barking when they see other dogs out the

window. I understand.

Well, this model just gave me an

understanding of something that I don't

think I otherwise ever would have

received on my own. So, for that, I am

thankful.

This is definitely much better. Can we?

Good. We can hit the car. Oh, and

there's still the replay feature. So,

let's try to do something sick and then

we'll This is definitely more in line

with what I would have wanted to have

seen. All right. All right. All right.

Don't bail. Yeah. Let's replay it

anyway.

How How long is this replay going for?

Oh, okay. [laughter]

This is a long replay. And it actually

has a bar on the bottom there, the

yellow one that's moving around that

actually shows us like the progression

of this clip.

Does it show the bales in the replay?

We're going to find out inevitably.

Okay. No, but that's all right. That VFX

cam, I have to say that is absolutely

cool. I've not seen any other model do

that before. And that does add something

to this that I do like. Somebody's

walking a dog. Here we have the fire

hydrant spraying.

Oh, okay.

You would think we would have run over

the dog, but nonetheless. So, this is

definitely better.

Now, we did have to give it a reference

photo from Sonnet, but I mean, those

results are absolutely monstrous. I'm

very pleased with this. It did take

quite a while, but just the ability to

replicate the image and then also

putting in the other skater there, the

the camera in the bottom right, the way

the pigeons fly away, really that camera

in the bottom right is kind of

making me quite pleased with this.

Overall, a significant significant

improvement when given a decent source.

Next up, we're trying the front-end

website test where it has to create the

beautiful watch website. This isn't the

one where it does the special edition

watch dubbed the BJN. I'm just more

focused on how it creates the watch, I

suppose, with the model and how it does

in some front end like this. This was

run with the full version of this model

on Mac. So, absolute highest potential

here. Let's take a look and see how it

did. at least judging this based off of

a front end result if nothing more.

Cool. It is in a single file. And this

isn't the one with the special edition

BJN watch. So, it's just like a normal

watch.

The only problem I'm noticing here,

well, aside from the strap not being

like the sideways um watch face is

letting it down, but it's interesting

because the numeral markers are properly

oriented. So, the 12 does have two of

them. So, it just made the face

sideways. Overall though, aesthetically,

this definitely is not bad.

Interestingly, I also noticed a crooked

watch face just in the subway FPS result

kept not chased. Okay, we have some

pretty bold front end. This is a decent

looking exploded view as well, I will

say. No movement on the emulated

movement, however, I would like to see

that, but acceptable. And it does have a

few individual pieces here. Let's see if

we have our other models as well. Good.

Much better. I would personally rather

have the 39

than Slappice. Okay, acceptable. Next

up, I gave it a follow-up prompt for the

subway FPS, just asking it to overall

improve the entirety of it massively. I

did not give it any reference photos, so

that's where this will differ from the

skate game improvement. Additionally to

that, this was run from within the same

session as when the result was created

from scratch. So, that was also a way

that this differs from the skate game

improvement. But let's just take a look

at this.

Is that better?

I see some difference in like Okay, it's

definitely better in that regard. All

right. And I didn't tell it to do that.

I just said make it better. Let's see if

our zombie enemies have changed at all.

Okay. One of them has like a hard hat,

but unfortunately they still have It

just added like some light bloom and

things like that. If I turn the

flashlight off, it's not as bad. It just

kind of restyled their outfits. It

didn't necessarily change their

design.

Oh, I think that's supposed to be a fan.

I think it's just kind of sideways.

Okay. I [laughter]

I didn't think that would happen, but

that's good attention to detail.

Let's just

Let's just do that one more time. All

right. Now, let's stand back from the

train.

We'll see if the interior of the train

is at all like

Is that a

NPC? It is. Okay.

Look, these handle things are actually

swaying when the train moves. That's

something I purposely pointed out almost

like kind of as a joke, just to be a

jerk. That was something I pointed out

that other models did not take the care

to do. So, that is an interesting little

attention to detail that I'm surprised

to see here and happy to see. We also

have some arrival boards for the trains.

Delayed disturbance. Okay. All right.

And this is pretty much like the same

overall. It did add some graphical

improvements and it didn't take super

long for this to be done. So, an

improvement. Did it happen again? Ah,

okay. So, they did fix the usage glitch.

It's just you don't you don't get a ton

of usage. So overall, that is going to

conclude our first look and test of Muse

Wonder. Interestingly, it seems like the

usage just reset even from when we

looked at it a second ago and saw it was

tapped out. So, if there are some issues

with this, I'm then going to refrain

from commenting on the usage for the

price of the subscription because it's

not an accurate reflection of what it

will end up being. That's fine. I would

say overall these are definitely showing

some strength. The result that sticks

with me the most is the second iteration

of the skate game where we gave it a

reference photo and it really did a

fantastic job replicating that. Even

adding in that secondary camera of I

suppose I should just show what I'm

talking about right here. So like this

second camera right there from the like

thing that's recording us for the

replays. That was absolutely awesome. I

very much liked this and this did a

significantly better job when given this

reference photo even having other

skaters. something I've never actually

seen before. So, it definitely has some

capability because yes, I gave it a

reference photo that I think Sonnet had

done. However, if this model was not

good, it would not have been able to do

a nice job in replicating what was seen

in that photo. So, I'm happy to see

that. And then this was pretty cool, the

replay as well. And overall, this made

me bullish for this model. Additionally,

I do have to say in the case of the I

won't say smaller of the two because I

don't know, but in the case of the more

accessible and less performant of the

two, this Muse Wonder model seems to be

pretty token efficient. Just as a first

impression, it was finishing its results

pretty quickly. Even on the highest

reasoning effort that it has, I do have

to say though, again, the benchmarks

right here are putting both of these, so

these are the two new Muse Wonder

models. These are putting these both

leagues ahead of some of the other ones

that are listed right here. And I

understand that it may just not be as

good as doing aesthetically pleasing 3D

stuff. I mean, Anthropic is planning to

potentially IPO. They probably want to

make sure those models look the best in

the oneshot generations that get shared

on social media and get probably some of

the widest reach leading to first

impressions of the model. I understand

that even if I'm not showing this doing

like intricate bug fixing on camera and

things like that, I understand there are

differences in terms of what the models

are trained to excel at. However,

personally, as someone who sees a lot of

models and tests a lot of models, when

something comes out with benchmark

scores listed right here, it gives the

impression that this is going to pretty

much mop the floor with everything else

right here, including those oneshot

results. And this is not the case here.

I find a lot of folks online were

mentioning that they felt some

disappointment based on first results

for this. There were even some mentions

of this by folks working at Meta saying,

"Well, we didn't really train it to

excel at 3D." I get that. But I will say

in that announcement blog post that we

first looked at, one of the prominently

listed things was a game where that

little Muse agent was cooking in a

bakery or something like that. So, you

know, I mean, make of that what you

will. And that's why I like running

these tests because we have a ton of

reference point for how other models

have performed in the same exact tests

and then that allows us to basically

objectively view the results and see

what we think. Overall, I love seeing

Meta actually competitive. I think these

models are actually pretty good and I

will be honest, I think they come off

worse than they are. Specifically in the

case of this one right here where it was

token efficient and it seemed pretty

smart. I will say though at $4 per

million in and $20 per million out, I

think it's too expensive when comparing

to both this and okay, neither of these,

but that is the same price as this model

right here. And GPT61 Soul is cheaper

than that and is a powerful model even

if it's kind of like not good company.

However, positives about Meta, they are

the only ones who have like decent

openweight models. Well, at least out of

themselves, XAI,

Anthropic, and OpenAI. OpenAI had GPTO

OSS, and those were very good when they

came out, but that was quite a long time

ago. Meta has Muse Glimmer, and they do

also talk about potentially open

sourcing some of the Spark models. So,

they're still in very good standing, at

least in terms of that, and I do very

much appreciate that. So, these models

are polarizing and I will be interested

to see some of the results that get

showcased with these when a little

additional time has passed. So, overall,

I think that is going to conclude our

first look and test of Meta's Muse

Wonder. If you have any questions,

please feel free to leave them in the

comments. Hey, that's kind of cool. And

thanks for watching.