Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Jeroen]

Hi Jan,

How will you implement the silence detection we spoke about, coz my question is if you are able to read the volume from a sample in real-time?

Secondly, I mailed Toon Boom studios, and they responded positive to the suggestion it should be possible to export a lip sync text file just like Pamela. Cool huh? And I always think larger developers do their best to avoid being mailed :-D

[Mnemonic]

How will you implement the silence detection we spoke about, coz my question is if you are able to read the volume from a sample in real-time?

Well, probably not in realtime. I'm leaning more to the .talk file approach (i.e. an external file describing the sound characteristics, including the pauses).

Secondly, I mailed Toon Boom studios, and they responded positive to the suggestion it should be possible to export a lip sync text file just like Pamela. Cool huh? And I always think larger developers do their best to avoid being mailed :-D

That's a great news!

[Jeroen]

Yes, but if it isn't realtime I'd have to adjust and browse through approx 1000 samples of samples - silence detection IMHO should be an exception where you enter something like this in the actor definition:

SilenceDetectionMinVolume = 10
SilenceDetectionMinDelay = 30
SilenceSprite = mySprite.sprite
// when the volume drops below 10 for 30ms the sprite reverts to SilenceSprite.

OR if this will be a performance dropper, a TALK editor tool should be smart enough to detect silence parts in a sample.

[Mnemonic]

OR if this will be a performance dropper, a TALK editor tool should be smart enough to detect silence parts in a sample.

Yes, that's what I had on my mind. IMHO it's not feasible to do it in realtime

[Jeroen]

Batch conversion all the way!  

[creatorbri]

So if I understand correctly, a separate utility will read through/play through a sample, detecting speech volumes, and compile a sort of speech-sync map file?

[Mnemonic]

Yep, that's the goal. Support for those "map files" is already present in the current version, but it's not officially released/documented yet.

[McCoy]

Sorry to bump up this old thread, but I wanted to ask (I already made it in the WME blog, but I thought it would be better if I post in the forums):

Jan, now that 5MA is going for voice acting, are you thinking about adding silence detection and lip sync to the engine? Specially silence detection. Any improvement about the approach you mention in this post? Still no way of making silence detection realtime without significant performance lose?

By the way, lip sync is not that necessary in my opinion, as it would be a lot of work both for the user and for you, so maybe that would fit for a long-term task, or for the next-gen tools maybe. But silence detection is basic for voices in a game imho.

[Mnemonic]

No, real-time is still a no-go, I'm afraid. There are several reasons. The sound decoding is done asynchronously and it would be pain to match the timing with animations. Plus, it needs to be fast and this would require some sort of sampling and interpolating of the sound.. The "map" files mentioned above will always be the primary way of attaching animations to sounds, but of course their creation will need to be supported by some tool.
There's no lip-sync planned for 5MA at the moment.