The Dukes (19-9, 10-5 Sun Belt) led 40-30 at the break, and led by as many as 13 on three occasions in the second half, the last being on a Julien Wooden jumper with 9:37 left for a 57-44 advantage.
The lead was still 12 points with eight minutes left on a Wooden three-pointer.
ODU (16-11, 8-7 Sun Belt) found a spark as Chaunce Jenkins hit a free throw, then a jumper followed by a Jason Wade layup as the Monarchs cut the lead to 60-53 at 6:33.
The lead was back at double figures, at 65-55, following a pair of Mezie Offurum free throws with 4:33 remaining.
The Monarchs found one more run, going on a 10-2 spurt capped by the fourth three-pointer of the night from Baker, to trail 67-65 at the 2:08 mark.
From that point, JMU hit 9-of-10 from the line to take the win.
“Any time you lose its not great, but I am certainly proud of how our guys battled being undermanned, scrapping and finding a way to come back against an outstanding JMU team,” ODU coach Jeff Jones said.