There is much discussion, and much controversy, on what constitutes "Down East". To New Yorkers and Bostonians, and probably the rest of the US, Maine itself is Down East; but no. Within Maine, in reality, only part of Hancock County and most of Washington County are Down East ... not counting estuary waters, a land mass of about 2500 square miles (6500 sg km) (and bigger than the state of Delaware and twice as big as Rhode Island.) Tourism boards include Mt Desert Island (Bar Harbor and Acadia National Park) as part of Down East, but also no: they are very much too gentrified and pretentious to be included in this blue-collar, hard-scrabble region.
The appellation "Down East" comes from sailing ship period in colonial America, when downwind was east along the coast.