Yeung replaced John Tong, who – at 78 – served three extra years as bishop with permission from the Pope. Yeung, 71, was officially appointed by the Vatican on Tuesday evening.
In recent years, the Communist state has ordered the removal of Christian crosses from religious buildings across China.
“I believe, at the outset, it was done by some provincial officials, then political factors were involved as it continued – the issue became bigger and bigger,” he said at his first press conference.
“But I believe the issue is very complicated – is it related to building safety? Maybe… But before the demolition, did people discuss it well enough? Maybe that is the issue.”
He said he would not want the crosses at his cathedral to be removed, if they compiled with building safety regulations.
“But if there are an illegal structure and the government has to remove them, I can’t say we are always right. In society nowadays, we need good communication.”
Asked by reporters about his views on vindicating the 1989 democratic movement in China, he said that the June 4 massacre was an unfortunate incident that made him sad.
“But I am a realistic person. If you know something cannot be done now, if you know it is a thick wall, do you still bang your head against towards it? I would not,” he said.
“If you ask me would I support the students at the time – I would, until today. They selflessly gave their whole lives, fighting for people’s rights, equality, justice, integrity.”
Likewise, he said he felt sadness for the late Chinese dissident Liu Xiaobo.
“He basically died of illness in prison,” he said. “But the incident has happened – do we just sit down and cry, and not stand up again? I am a realistic person, I cannot do much on this issue, are there any other things that we can do?”
On potential diplomatic relations between mainland China and the Vatican, Yeung said: “I really don’t think Hong Kong has a position to take, [or] has anything to say.”
“However, we are willing to be a bridge – I always think that building bridges is more important than building fences,” he said. “If there is any way we can help to maintain a dialogue, we will be willing to do so.”
He added that the church was often in touch with the mainland, often training new talent regardless of government recognition.
Former bishop Joseph Zen was an outspoken critical voice with regards to the potential relationship.
“I don’t believe Cardinal Zen opposed diplomatic relations between China and the Vatican. He only said we should not kneel down easily,” Yeung said.
But he added that only the Pope can appoint bishops, and not any other power: “This has never happened anywhere in the whole world.”
Yeung said the Vatican only established diplomatic relationship in the 1980s with London and Washington.
“As there are still things to discuss, it will not be done overnight,” he said. “I will do whatever I can do.”