Yes, that was it.

It was the shadow of another face in his that had startled her with so strange a feeling, almost as if she had been looking upon some ghostly thing.
Another face, like and yet unlike.
But what face?
whose face?
She could not answer that question, and her inability to solve the enigma tormented her all tea-time, as the stranger sat opposite to her, making a pretence of eating heartily, in accordance with Mr.Whitelaw's hospitable invitation, while that gentleman himself ploughed away with a steady persistence that made awful havoc with the ham, and reduced the loaf in a manner suggestive of Jack the Giant-killer.
The visitor presently ventured to remark that tea-drinking was not much in his way, and that, if it were all the same to Mr.Whitelaw, he should prefer a glass of brandy-and-water; whereupon the brandy-bottle was produced from a cupboard by the fire-place, of which Stephen himself kept the key, judiciously on his guard against a possible taste for ardent spirits developing itself in Mrs.Tadman.
After this the stranger sat for some time, drinking cold brandy-and-water, and staring moodily at the fire, without making the faintest attempt at conversation, while Mr.Whitelaw finished his tea, and the table was cleared; and even after this, when the farmer had taken his place upon the opposite side of the hearth, and seemed to be waiting for his guest to begin business.
He was not a lively stranger; he seemed, indeed, to have something on his mind, to be brooding upon some trouble or difficulty, as Mrs.Tadman remarked to her kinsman's wife afterwards.

Both the women watched him; Ellen always perplexed by that unknown likeness, which seemed sometimes to grow stronger, sometimes to fade away altogether, as she looked at him; Mrs.Tadman in a rabid state of curiosity, so profound was the mystery of his silent presence.
What was he there for?
What could Stephen want with him?
He was not one of Stephen's sort, by any means; had no appearance of association with agricultural interests.

And yet there he was, a silent inexplicable presence, a mysterious figure with a moody brow, which seemed to grow darker as Mrs.Tadman watched him.
At last, about an hour after the tea-table had been cleared, he rose suddenly, with an abrupt gesture, and said, "Come, Whitelaw, if you mean to show me this house of yours, you may as well show it to me at once." His voice had a harsh unpleasant sound as he said this.

He stood with his back to the women, staring at the fire, while Stephen Whitelaw lighted a candle in his slow dawdling way.
"Be quick, man alive," the stranger cried impatiently, turning sharply round upon the farmer, who was trimming an incorrigible wick with a pair of blunted snuffers.