Htydjtk

In Star Trek: Enterprise you view various clues to how the translator is built. Hoshi Sato, supposed linguistic genuis, meets various alien races, and builds up the translator by adding in alien grammers combined with a speech processor and voice samples of the aliens language.

The translator would use AI, probably artifical, and similar to what is available in Google translate, (but much better, as it's set in fictional universe, but close it will be in this time frame) from a starting point, take a speech sample from the new alien, and use different search algorithms to match, on different alien database to find a similar or close matching grammer. From the grammer, the speech processor can take the grammer and enunciate possible speech patterns that the new aliens may understood, and by iterating down the search tree, zero in on a closer, possible better match, as they get more speech back from the alien.

But it would break, if non of the grammer was even a close match. Imagine two alien species seperated by say, 2000 light years. Their speech would be a different as basque, say one of those languages on earth, where you click the tongue, like tagalog.

Very similar to the way IBM recent computer match against the Jeopardy winners would works. Using search algorithms to map broken down question terms to possible answers.

In the Darmok episode of TNG, we get an insight into how it works.

Essentially, when encountering a new language, the universal translator takes samples of the new language and compares to known languages, and slowly builds up a database of words and phrases. In the episode linked, the UT has trouble understanding the alien language because they speak mostly in metaphors.

See also Universal translator for a more in-depth explanation, and more examples of episodes which feature the UT.

You might as well just think of it as magic. Like many other devices in Star Trek (e.g. the transporter), it's there for plot reasons rather than being based on any kind of scientific or technological extrapolation. In the case of the universal translator, it's so that they don't have to spend the first half of every episode with aliens on basic language lessons.

The same way that most codes are broken in real life. It would search for frequently repeated words and sounds, and figure out what words would be used with the same frequency in the host language. Star Trek Corps of Engineers explained it a little more than in the series.

Copied from my answer in If there are universal translators, why are some words/phrases not translated?

From http://en.memory-alpha.org/wiki/Universal_translator (Which got it from TOS 2x02: Metamorphosis)

Responding to Zefram Cochrane's question about the theory of
operation, Kirk explained that there are certain universal ideas and
concepts common to all intelligent life, and that the translator
compared the frequencies of brainwave patterns, selected those ideas
it recognized, and provided the necessary grammar. Kirk further
explained that the device spoke with a voice, or the approximation of
one, that corresponded to the identity concepts it recognized.

Since it's already scanning brainwaves, presumably it can also detect intent (whether you want to be understood or not), and decide whether or not to translate as appropriate.

Plus my comment on that answer, addressing Darmok and similar languages

You're assuming their brainwave patterns were similar to general humanoids.... Like how Betazoid's can't read Ferengi because of their brain structure, there's a good chance the UT couldn't correctly interpret the Darmok aliens' thoughts because it had nothing like that to compare to.

I always felt that the Star Trek Universal Translator worked directly with the brainwaves. It was similar to Farscape’s translator microbes, it makes you feel as if the other person was speaking in your language. This is actually shown in an episode of Discovery, we hear the Klingon speaking in Klingon with subtitles and once the UT is activated we hear them in English (or whatever human language the dub is in), the Klingon even says “I did not expect you to speak Klingon” and Michael says it is the translator. This would also explain why the Japanese heard people speaking in Japanese in “The 37” and how Picard and Data are capable of going undercover as Romulans into Romulus itself. You don’t really hear anything over the other person’s voice, you hear the person’s voice in your language because the trick is that your brain itself changes the meaning inside your head.

Of course, this doesn’t explains the lip-sync, technically that should still be an issue. But everyone who has seen a dubbed movie (and I don’t mean the terrible bad old times dub of Asian martial art films, but the careful quality dubbing you can see in Western media like between Spanish, English and French) your mind ignores the lack of sync. I guess it could be hand-waved away that unless the language is too different and it takes you a lot of time to say one word your mind just overlooks the lack of sync like watching a dubbed film.