Machine Translation Using LSTMs.

Model’s Architecture: 2 Layer Uni-LSTM encoder, 2 Layer Uni-LSTM decoder, ADAM optimizer and Cross Entropy Loss function. Model’s METEOR Score=0.245, BLEU Score=0.0066. In the first phase, I started understanding the general concepts of NMT by implementing the model from “Sequence to Sequence Learning with Neural Networks” paper.

Machine Translation Using Bidirectional GRUs.

Note: One negative of the Phase 1 model was that the decoder was trying to put lots of information into the hidden states. While decoding, the hidden state require to contain information about the whole of the source sequence, as well as all of the tokens that have been decoded so far.

Model’s Architecture: Bidirectional-GRU Encoder, GRU Decoder with Attention mechanism, ADAM optimizer and Cross Entropy Loss function.Model’s METEOR score=0.356, BLEU Score=0.0687.

I used single layer GRU in the Encoder, which was a bidirectional GRU. With a bidirectional GRU, we have two GRUs in each layer. A forward GRU going over the embedded sentence from left to right and a backward GRU from right to left. I only passed an input (embedded) to the GRU, which tells PyTorch to initialize both the forward and backward initial hidden states to a tensor of all zeros. I also got two context vectors, one from the forward GRU after it has seen the final word in the sentence, and one from the backward GRU after it has seen the first word in the sentence.

An instant later, a symphony which had been intended to produce the greatest effect on the arrival of the Virgin, was lacking. Gringoire perceived that his music had been carried off by the procession of the Pope of the Fools. “Skip it,” said he, stoically.

He approached a group of bourgeois, who seemed to him to be discussing his piece. This is the fragment of conversation which he caught,—

“You know, Master Cheneteau, the Hôtel de Navarre, which belonged to Monsieur de Nemours?”

“Yes, opposite the Chapelle de Braque.”

“Well, the treasury has just let it to Guillaume Alixandre, historian, for six hivres, eight sols, parisian, a year.”

“How rents are going up!”

“Come,” said Gringoire to himself, with a sigh, “the others are listening.”

“Comrades,” suddenly shouted one of the young scamps from the window, “La Esmeralda! La Esmeralda in the Place!”

This word produced a magical effect. Every one who was left in the hall flew to the windows, climbing the walls in order to see, and repeating, “La Esmeralda! La Esmeralda?” At the same time, a great sound of applause was heard from without.

“What’s the meaning of this, of the Esmeralda?” said Gringoire, wringing his hands in despair. “Ah, good heavens! it seems to be the turn of the windows now.”

He returned towards the marble table, and saw that the representation had been interrupted. It was precisely at the instant when Jupiter should have appeared with his thunder. But Jupiter was standing motionless at the foot of the stage.

“Michel Giborne!” cried the irritated poet, “what are you doing there? Is that your part? Come up!”

“Alas!” said Jupiter, “a scholar has just seized the ladder.”

Gringoire looked. It was but too true. All communication between his plot and its solution was intercepted.

“The rascal,” he murmured. “And why did he take that ladder?”

“In order to go and see the Esmeralda,” replied Jupiter piteously. “He said, ‘Come, here’s a ladder that’s of no use!’ and he took it.”

This was the last blow. Gringoire received it with resignation.

“May the devil fly away with you!” he said to the comedian, “and if I get my pay, you shall receive yours.”

Then he beat a retreat, with drooping head, but the last in the field, like a general who has fought well.

And as he descended the winding stairs of the courts: “A fine rabble of asses and dolts these Parisians!” he muttered between his teeth; “they come to hear a mystery and don’t listen to it at all! They are engrossed by every one, by Chopin Trouillefou, by the cardinal, by Coppenole, by Quasimodo, by the devil! but by Madame the Virgin Mary, not at all. If I had known, I’d have given you Virgin Mary; you ninnies! And I! to come to see faces and behold only backs! to be a poet, and to reap the success of an apothecary! It is true that Homerus begged through the Greek towns, and that Naso died in exile among the Muscovites. But may the devil flay me if I understand what they mean with their Esmeralda! What is that word, in the first place?—‘tis Egyptian!”