The day after Harvard announced that Polish labor leader Lech Walesa had accepted an invitation to speak at the University's Commencement exercises, three separate press dispatches from Europe yesterday reported that Walesa does not plan to make the trip to Cambridge.

But University officials continued to stress that in the absence of any direct turndown from Walesa, they are still holding out hope that he will deliver the June 9 Commencement address.

The Associated Press yesterday quoted Walesa as saying. "I'd love to go," and adding. "But my decision today is no, because of the problem in returning."

In addition, a spokesman for Radio Free Europe in Washington, D.C., said the agency had aired a dispatch in which Walesa's wife. Danuta, said she planned to notify Harvard today of her husband's decision not to travel to the United States.

In the report, which the radio agency attributed to the Reuters news service's Warsaw bureau. Danuta Walesa cited her husband's fears that Communist authorities would prevent him from returning to Poland, the spokesman said.

The AP also released an earlier interview with an unidentified woman who said, from Walesa's Gdansk home. "He will be unable to travel there for well-known reasons.... He is simply not sure that he would be able to cross the Polish border in both directions.

Harvard Remains Hopeful

But Robin Schmidt, Harvard's vice president for government and community affairs, said yesterday. "We're still hopeful that he'll be our Commencement speaker." Schmidt said that Harvard's only direct word from Walesa so far is a March 5 letter that states, in Polish, his acceptance of Harvard's invitation.

"We've had more than one confirmation" of Walesa's acceptance, Schmidt added, refusing to explain how Harvard received its supplementary information about the labor leader's intentions. But as long as Harvard has no further word from Walesa, the University is not giving up on the possibility that he will come to Commencement Schmidt said.

At that time of Schmidt's comments, the only press reports from Poland about Walesa were the interview with the unidentified woman at Walesa's home and a United Press International bulletin Thursday in which he reported said, "No, I am not going.

Walesa was released in November from confinement that began a year earlier, when the Communist government imposed martial Law in Poland.

Polish specialists have said that Gan. Wojciech Jaruzelski's regime is probably eager to see the 39-year-old electrician emigrate from his homeland, thus weakening his still-powerful influence on Polish workers.

The AP also quoted Walesa as saying that his letter from Harvard mentioned the possibility that the University would grant him an honorary degree, adding that he would be pleased to receive it by proxy.

Told that Harvard policy requires honorary degree recipients to attend Commencement exercises. Walesa said, "So I would be the first" to break with the tradition. He added.

I'd love to be the first, although I'm really sorry I most probably won't be able to be there in person," the AP reports.