He said three staff have retired and around eight staff have left in the past 18 months.
OPW Minister Kevin Boxer Moran has claimed that OPW staff are being "bullied" at Castletown House.
He said he examined a letter given to him by staff members.
He said the letter handed to him outlined how staff feel and he "can't believe" some of the aspects staff highlighted.
He said staff said they are being filmed going to work or if they "go for a drink at night" they are being bullied or contacted and that they don't "feel safe".
He said three staff have retired and around eight staff have left in the past 18 months.
He said he has received a letter from the Irish Wheelchair Association outlining concerns about accessibilty on Lime Avenue.
He said "all OPW sites" are shared entry and "work extremely well".
Speaking on national radio, he said putting traffic control on the avenue at a cost of €10,000 per week is not "practical".

Kildare Filling Stations Among The More Than 100 Out Of Fuel
Cost Of Solar Panels In Kildare Falls By 39% Since 2020
Water Outage In Place For Maynooth And Straffan Tomorrow
Kfm Obituary Notices
Nationwide Fuel Protest Continues Into Third Day
"I Don't Think Anybody Is Going To Really Listen To Us Until Somebody Dies On That Road": Criticism Of Moone-Timolin Cycleway Continues
International Wholesale Prices Are "Not Price Gouging", Says CCPC
Costs Of Home Heating Oil Up By 67.5 Per Cent Between February and March