He said three staff have retired and around eight staff have left in the past 18 months.
OPW Minister Kevin Boxer Moran has claimed that OPW staff are being "bullied" at Castletown House.
He said he examined a letter given to him by staff members.
He said the letter handed to him outlined how staff feel and he "can't believe" some of the aspects staff highlighted.
He said staff said they are being filmed going to work or if they "go for a drink at night" they are being bullied or contacted and that they don't "feel safe".
He said three staff have retired and around eight staff have left in the past 18 months.
He said he has received a letter from the Irish Wheelchair Association outlining concerns about accessibilty on Lime Avenue.
He said "all OPW sites" are shared entry and "work extremely well".
Speaking on national radio, he said putting traffic control on the avenue at a cost of €10,000 per week is not "practical".

Kfm Obituary Notices
Sharp Rise In M9 Collisions With Garda Call-Outs Up Year-On-Year
Councillor Seeks Clarity On How Often Nightclubs and Late Bars In Naas Are Inspected For Fire Risks
Unpaid Eight-Month Work And Training Placement For Adults With Disabilities Branded "Extremely Unfair"
Department Of Education Defends Kildare SNA Cuts Amid Claims Of Reduced Need At Some Schools
Naas Nurses Threaten Escalation As Staffing Row Deepens
Driving Tests Face Disruption As RSA Testers To Strike Next Week