The announcement marks the latest push in a sweeping drive to root out graft at all levels of the party and state since President Xi Jinping came to power more than a decade ago.
“Following a review… it has been decided to initiate an investigation into Zhang Youxia and Liu Zhenli,” the defence ministry said in a statement.
The two men were “suspected of serious violations of discipline and the law”, the ministry said.
Zhang Youxia, 75, is China’s highest-ranked general as the more senior CMC vice chairman.
He is also a member of the powerful Politburo, the 24-member executive body of the Chinese Communist Party.
He shares the vice chairman title with Zhang Shengmin, a general in Beijing’s secretive rocket force and who is no relation.
Zhang Shengmin was promoted to the post in October after Beijing expelled his predecessor in another sweeping corruption purge.
Both generals rank below Xi, who has held the CMC chairmanship since 2012.
Liu, 61, is the chief of staff of the CMC’s joint staff department, which oversees combat planning.
Rumours of a probe swirled this week after Zhang Youxia and Liu appeared to miss an official meeting chaired by Xi and attended by second-ranked CMC vice chairman Zhang Shengmin.
Xi has called graft “the biggest threat” to the Communist Party and said “the fight against corruption remains grave and complex”.
Proponents say the policy promotes clean governance, but others say it also serves as a tool for Xi to purge political rivals.