The paper describes an approach taken to estimate the probabilities of failure associated with various railroad tasks to prevent accidents (principally collisions and derailments). These probabilities were estimated using an expert elicitation process that used partially relevant data available from a variety of databases and that were filtered and scaled to make them more directly relevant to the analyses being performed. Extensive qualitative studies were performed prior to the elicitation process to identify relevant contexts under which the tasks can be performed.