Kevin Krebs, 31, was taken into custody in the 100 block of High Street on Saturday.
The pipe bomb was discovered when investigators went to Krebs' home in Malvern, Pa. on Monday.
Police arrest armed man during 'No Kings' rally in West Chester, Pennsylvania
Once it was discovered, police backed off their search and got out of the house.
Alex Krebs, who said Kevin Krebs is his younger brother, spoke to Action News right on the front porch before the bomb squad and FBI arrived.
"Completely so apologetic, and so sorry that everybody was triggered or people felt uncomfortable and unsafe," said Alex Krebs.
According to the affidavit of probable cause, police seized a fully loaded handgun, three loaded handgun magazines, six loaded assault rifle magazines, knives, a ski mask and gloves from Kevin Krebs at the time of his weekend arrest.
He faces charges for failing to obtain a concealed carry permit, and more charges are possible.
Alex Krebs says his brother was actually in downtown West Chester in support of the protest and its message.
Investigators say Kevin Krebs told them he was armed to protect himself after seeing violence at protests elsewhere in the country.
"In his brain, he's scared, but I always tell him, 'You are safe, always,'" said Alex Krebs.
After Kevin Krebs was arrested Saturday, we're told the FBI searched his room and social media but found nothing.
However, they were back on Monday with a search warrant for the entire house and found the pipe bomb. It's not known where in the house the bomb was found.
RELATED: Police arrest armed man during 'No Kings' rally in West Chester, Pennsylvania
As this investigation unfolds, Alex Krebs said his brother is "harmless" and "not a threat."
"We're going to properly get him in the right direction to seek professional help," he said.
Kevin Krebs is out on bond but his current whereabouts are not clear.
The bomb squad remained on the scene as of late Monday night.