The building was originally built in the year 1887 around a beautiful garden and designed as Manor House. The property was originally owned by the Maharaja of Cooch Behar and leased to a succession of British families such as the Pavion and Oakley families. It was originally a club and then remodelled as a hotel. In the year 1965, Kuldip Chand Oberoi, of the Oberoi family of Hoteliers, bought the hotel from the Maharaja of Cooch Behar and Nancy Oakley. Nancy returned back to England after the handover.
Legend has it that the name "Elgin" was derived from the village of Elgin in Scotland. The hotel has been restored to its original grandeur. The snug and cozy interiors are decked with etchings of G.Douglas, lithographs of Daniell, period Burma teak furniture, oak floor boards and paneling, crackling fireplaces, candle-lit tables and strains of music from the grand piano. One realizes the meaning of palpable 'old world charm' when one visits this charming colonial hotel which has played host to dignitaries like the US Ambassador and the Crown Prince of Sikkim to Dominique Lapierre and Mark Tully. The hotel has stories to tell from the time of its first owner the Maharaja of Cooch Behar in 1880s to the present owner Diamond Oberoi spanning over a century of memories.