Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[skyPhyr]

Hi Mnemonic,

As you'd be aware this is my first post here. I'll say this breify so I can get to the point. You've done an extremely professional job of putting wintermute together. The interface holds up better than many apps I've used professionally in multimedia development.

Sorry to start with a feature request, but this is something which I noticed which I think would be of huge benefit. It involves two parts.

The first is to create a text definition type whereby your talk function could pass a null for the text which signifies that the text is contained in a file the same name as the sound file (with a different extension of course). This file could contain not just the text, but synchronization information.

The synchronization information could contain two parts. The text synchornization. i.e. when each sentence appears. As well as basic phoneme synchronization.

The phoneme synchronization would also require some actor expansion to be useful. i.e. the ability to configure different phoneme sprites which are then driven by this phoneme synchronization data during talking.

I hope this makes sense and doesn't sound unreasonable. I've no idea how your code is structured so these suggestions might be completely unrealistic.

Would love to hear what you think.

Cheers,

Alan.

[Mnemonic]

Hi Alan,

thanks for your suggestion. It certainly *is* reasonable. In a matter of fact we've been discussing this with Jeroen (Project Joe designer) some time ago. Basically, WME is ready for this, it supports the "talk stances". If the talk stances were actually the phonemes, it would work. But of course, the worst part is creating lips-sync definition for the sound files. So far I was only thinking about some very basic lips-synchronization based on the sound volume.

One think's for sure, it would need to be supported by some tools, because there would be *lots* of the sync-definition files. Actually, I believe the Gabriel Knight 3 game uses a very similar system and there was somewhere about 8,000 of the speech definition files.

But I'm definitely adding this to my (ever growing) TODO list

[skyPhyr]

Thanks for the reply. If you like I'm happy to look through your current code and see if I can come up with complimentarily structured definition for these text files.

I haven't really done much "hard-core" programming. (Read that I'm a multimedia developer - so only done a lot of Java/JScript, PHP, but I'm predominantly a graphically articulate person)

Though I'm still more than happy to put the research in and develop a tool to create the sound files if you'd like a hand. Though you'd probably need to rewrite it afterwards to integrate it properly.

Anyway - anything I can do to help - just let me know.

Cheers,

Alan.

[Mnemonic]

OK, like I said, the worst part is analyzing the sound file and converting it into a bunch of phonemes, so any information on this topic would be really helpful (links, algorithms etc.). The rest would be relatively easy (I think )

[skyPhyr]

ahhhh ok - I was thinking much simpler than that.

i.e. an application which simply allows someone to scrub through the sound file an add entries to a seperate expressions file.

So the file is basically manually created. This would also have the advantage of the person creating a custom stance which allows them to put in non-phoneme things like a scream or different type of ooooo's depending on how much pleasure/pain the character is in.

Would mean it would take more time for the person creating the game, but would be more flexible and reliable.

Would also integrate the on-screen text to the file so they can add either a stance or a on-screen text update at any time point.

Cheers,

Alan.

[Scarpia]

This is probably the most mind-puzzling thread ever, at least to someone like me, whose native language is not english, and to whom words like 'stances' and 'phonema' make little sense.

But I'll state my opinion on the matter anyways. I've pretty much covered every other thread on the board anyways  

I thought of a way to do lip-synchronization a while back, and the basic lip movements needed, I think, are:

"Openers" Bah Bee Boo Poe Bore Bay	Vowel sounds A (Ah) E (Eek) E (Chew) O (Oh) O (Door) E (Hey)
More "Openers" Tar Tee Too Toe Tore Day	More "Openers" Far Fee Foo Foe Fore Faye
"Closers" ..sch ..m ..ph	...

A maniac would also add:

• smiles
• frowns
• screams
• pouty lips
• laughs
• sneers
• coughs
• burps
• whispering
• whistling
• dribbling
• yawning
• panting
• twitching
• etc.

Scarpia

[skyPhyr]

Hi Scarpia,

That's pretty much exactly what I'm thinking, but where you could set up a custom set for each actor. Which you can then use in your synchro files.

Combining this with the text display on the screen I thought was just a nice way of streamlining it.

Cheers,

Alan.

[Scarpia]

I just spent the last couple of days playing Broken Sword II through for ideas, and I'll soon be halfway through Broken Sword I as well. I think I know what was the inspiration for the look-and-feel of Wintermute's interface...  

Anyways, now I think I understand what the question was also about: Talking to characters in BS1+BS2, I noticed that they actually move a LOT while talking, and that their moves are beautifully coordinated with the context of what they're saying. E.g. when the kid outside the Irish bar in BS1 speaks about something embarassing, he looks at the ground and scratches his shoe against the pavement. That's really cool. I suppose this is the kind of thing that can be done with WME's "Talk Stances".


Scarpia

[Mnemonic]

I just spent the last couple of days playing Broken Sword II through for ideas, and I'll soon be halfway through Broken Sword I as well. I think I know what was the inspiration for the look-and-feel of Wintermute's interface...  

I admit that  

I suppose this is the kind of thing that can be done with WME's "Talk Stances".

Exactly, that was the idea. Still, the talk stances are too cumbersome, that why a tool is needed to simplify the task.

[sychron]

You could use your volume based approach, if you bandpass the sound file before taking the volume. You must know about the speakers charakteristic vowel formant frequencies (developers could provide them in the interface), and then bandpass the file for theese frequencies and get the volume from there. Effect would be a simple vocoder sample array, just concentrating on the formant frequencies. This Array can be used to select a proper stance.
Since the vowels are the most prominent parts when looking at talking persons, this simplified approach might perform quiete good.

[Mnemonic]

This thread is very old. If anyone's interested in possible lip-sync solutions, see this recent thread: http://forum.dead-code.org/index.php?topic=2106.0