public void AnalyzeDocument()
{
//arrange
var collection = new CollectionRequest
{
id = "2d2e7341-a3c2-4fb4-9d3a-779e8b0a5eff",
documents = new[]
{
"The drive for ever-greater institutional efficiency places an enormous burden on those involved in developing regulatory submissions. Jim Reilly suggests how this process can be engineered to function more effectively and meet the organisation's demands. Jim Reilly is a consultant, software implementation, for development partnering organisation Octagon Research Solutions Inc. He is based at the company's headquarters in Wayne, Pennsylvania, in the US. Email: [email protected] The pharmaceutical industry has been cast into the storm of the digital age in recent years. It is possible to walk around any pharma company and hear whispers of gene sequencing, predictive animal modelling, electronic data capture, XML authoring or the electronic common technical document. All of these are relatively new developments and form part of a trend towards the ever-elusive concept of institutional efficiency. It is clear that everyone wishes to do things better, smarter and faster. Those of us involved on the regulatory front have not turned a blind eye to this drive towards efficiency. Whether it be adverse event reporting (AERS in the US or EUDRAVigilance in the European Union), clinical trial applications/investigational new drug applications or marketing applications, we are confronted with a host of electronic means to submit data. The recent trend has been towards submitting data simultaneously across the globe. After all, what more efficient way is there to submit similar material than at the same time? Thanks to the advent of the common technical document, authors can pen the story of the safety, efficacy and quality of their therapeutic in a standard way. Gone are the days of writing similar content in alternative formats, or retrofitting information for a different purpose. In the drive towards commonality of review content amongst global regulatory agencies, regulatory groups in pharma have begun focusing their efficiency efforts towards simultaneous global submissions.",
"For the first time, a marketing authorisation application in the EU can be submitted at the same time as a new drug application filing in the US, thus driving organisational efficiency, longer patent exclusivity and, most importantly, making therapeutics available expeditiously worldwide. Due to this and other potential efficiencies, regulatory groups are facing increased pressure to deliver applications faster and to submit them to multiple agencies simultaneously. But with simplicity of delivery comes complexity of packaging. Indeed, the regulatory group has often been perceived as a packaging plant, where documentation is received from various contributing groups and compiled into a submission. However, that characterisation is, to put it politely, misguided. The process is a holistic continuum, requiring the orchestration of many cross-functional activities. It requires input from authors, clinical sites, data management, programmers and the regulatory group itself. The group must co-ordinate on-time receipt of internal and external deliverables, resolve issues stemming from these contributed deliverables, and is often under enormous internal pressure to dispatch the submissions to multiple agencies by a specific date. Saddled with this burden, the regulatory group very often faces managerial pressure to “produce” a submission as quickly as possible, even though the work of others may have resulted in its delay. To work within the increasingly complex milieu of the submission management process, it is important for those involved to be armed with detailed and accurate information. With the right information at their fingertips, this group can justify staffing, make accurate deadline projections and even institute process improvement. The onus is therefore on the regulatory group to institute a relatively new process concept: submission management.",
"Submission management can be defined as the “orchestration of the people, processes and technologies necessary to facilitate the production and management of global regulatory filings”. As the collection point for much of the output from functional groups, the regulatory group is in a unique position within the organisation as it can fulfil an informational need at both managerial and executive levels. Such a position permits a level of institutional process improvement to be driven. For example, with so many “moving parts” in a submission, it becomes difficult to synchronise processes and resources in such a way as to allow handoffs to exchange seamlessly. In order to ensure that deliverables are on time and “submission ready”, both the regulatory team and contributing areas have to agree upon a standard of deliverable quality, and to develop a mutually agreeable plan of delivery. In creating a system, defining what to measure and knowing how to measure, the regulatory group can effectively manage the complexities of the process while instituting a level of improvement for all contributors. Before assuming that a submission management capability can be instantly arranged, there are some fundamental tenets to follow. Define and implement a standard processFirst, it is an absolute necessity to define and implement a standard process for the development, compilation and submission of a filing. Without a standard, the process is not repeatable and apt to degrade and become problematic. Most organisations have a minimal “working” standard in place, but it is important to document a standard so that the process remains consistent and is transferable despite employee turnover. Additionally, a standard process reduces stress on team members. Instead of questions about how to perform one's job, questions should be reserved for more appropriate matters so team members can concentrate their efforts towards on-time delivery. Proper deliverable tracking should not focus only on the delivery of the final content.",
"It requires that the proper lines of communication be open amongst the regulatory and functional groups so that everyone is working towards the same goal and issues are resolved appropriately. Communication should also exist in the form of routine progress updates, thereby keeping all stakeholders focused on the common goal of a quality, on-time submission. Also, review cycles should be built into the process to confirm intent of document placement and navigation. After compilation, dispatch and archival, the regulatory group needs to be vigilant in monitoring any updates required by contributing groups or regulatory authorities, so that they may plan accordingly for future submissions. Determine what information is importantWith a standard process in place, the regulatory group can begin measuring in a meaningful way. Metrics provide information and are the foundation for “knowing” a process. They can identify holes or constraints and provide the capability to develop future projections based on historical results. Therefore, it is important to define the data necessary for measurement. Data such as the dispatch and approval dates are almost a given as points to track. However, if a regulatory group is to plan, execute and react towards the demands of a submission, it needs to track data more scientifically. Data can be tracked with regard to:projected submission dates;resource availability and utilisation;deliverable due dates (from authors, data groups, third parties, etc); anddeliverable returns to functional areas. From a planning perspective, it is important to plan a submission schedule. A longer-term schedule (one to two years) should be plotted for major applications and a shorter-term, floating schedule for smaller submissions. With a schedule in place, a regulatory organisation can align resources to accommodate the effort needed to meet submission deadlines.",
"From an operational perspective, this is important in order to manage personnel so that workload is balanced across the team and to ensure that staff are utilised appropriately. Plotting a submission schedule may reveal a spike at certain periods, which is a rampant circumstance in regulatory organisations, especially towards the year end as pressure builds to finalise the submission for agency review. A submission hastily sent at the end of the year may turn out to have originally been targeted for earlier in the year, perhaps when activity was not as brisk. However, any number of potential issues may cause the submission date to be pushed back until there is adequate resolution of the problem. Such issues abound in the submission development process, and are extremely important to track, both from an audit perspective and for the sake of process quality. Examples may be content-specific, where documents are often reversioned, even after being “approved”. Deliverables may slip deadlines or might be rushed due to deadline pressure, increasing the likelihood of inaccuracy or poor formatting. Issues might also be strategic in nature: problems with patient enrolment might delay a study, an update to manufacturing controls may be warranted or an agency query might require a response. Of these issues, deliverable quality and due dates are the most transparent and therefore easiest to track from a regulatory perspective. Deliverable tracking is a highly critical cog in terms of metric gathering as it allows for a view of deliverables in a time-sensitive fashion and those on a critical path can be tracked in relation to the overall submission deadline. When due dates are not met, especially for deliverables on a critical path, the submission deadline may be put in jeopardy. This places an enormous burden on the regulatory group to push the limits of their group to deliver a submission on time. Devise methods of measurementVarious techniques can be employed to track processes, with the prospect of learning and improving.",
"Deliverables can be tracked as a series of inputs, even before they are authored or created, by characterising or “typifying” according to the authoring, data, regulatory or third-party groups that produce them. These data can be tracked by noting the instances when deliverables must be returned to a contributing group for modification and should answer the questions of when, why and to whom the deliverable is returned. See Figure 1 for an example of document return rates by functional areas. Issues themselves can also be typified so as to standardise the types of problems an organisation encounters. Coupled with the deliverable types, it is possible to point to problematic trends related to specific functional groups or points in process. See Figure 2 for an example portrayal of submission issue rates by issue type. It is important to note that these data are not meant to point an accusatory finger at colleagues but are intended to illustrate how to make the process better for all. It is doubtful that clinical colleagues like having documents returned, just as regulatory colleagues would rather not pass the deliverable back. Figure 1. Percentage of documents returned by functional area during one month** Data gathered from three pharma companies and illustrated here in aggregate. Figure 1 depicts document return rates from regulatory back to various contributing groups. Whether the reason is content- or format-related, the deliverable is not “submission ready” when passed on to regulatory. Figure 2. Rates of issues in regulatory submissions** Data gathered from three pharma companies and illustrated here in aggregateIn Figure 2 the data portray the typical issues encountered through the course of an electronic submission, and their relative frequencies. This sort of information acts as a springboard into real process improvement.",
"An empirical case such as this can be used as a communication tool to upper management to effect change, whether that means better process control in a specific authoring group, or improved deliverable quality from a vendor. Develop or choose a solutionDepending on the group size, organisational need, and internal capability there are a variety of options for building a system to track the information. Spreadsheets are the de facto tracking tool and serve well enough to track the status and date information related to submissions and their deliverables. However, over time, spreadsheets can become unwieldy when tracking large amounts of data, and because they are literal files, data cannot be updated by more than one person at one time. An alternative option is to construct a database. Databases offer much more flexibility in terms of the type of data tracked, and can be developed to allow for ease of information entry through friendly interface screens. Because databases are not literal files, more than one team member can update information simultaneously. Also, from a reporting perspective, databases offer much more capability in the form of aggregating and analysing data. One downside to a database is that data entry is manually intensive, a process often referred to as “feeding the beast”. Last, for a robust tracking capability and higher performance, there are enterprise tracking and management systems. These systems offer more advanced capabilities for information capture and reduce the burden of information entry on team members. Implement an effective methodThe last step in instituting a good submission management capability is to effectively implement a system. Roles and responsibilities should be clearly defined, metrics need to be gathered and communicated in a repeatable way, and the vision should be sold up and down the organisation. It is important to communicate that the metrics gathered are not intended to place blame on a particular group for process inefficiency.",
"Metrics are meant to be revelatory with the prospect of encouraging improvement. In essence, the regulatory group is looking at the end product of much of the organisation's work through an analytical lens, and should promote improvement up and down the process chain. In addition, the regulatory group can use information to truly learn how its members do things. This way historical data can act as an accurate predictor of future activity. Instead of plotting a submission schedule based on arbitrary data or anecdotal information, the group can become more accurate with predictions as to when deliverables are received and how long it will take to get them ready for submission. If metric gathering is designed with the end questions in mind, a regulatory group, and indeed the larger organisation, will be well on its way towards improving its submission process."
}
};
//act
var responseId = _sut.SendCollection(collection);
var result = new CollectionAnalysis {
status = ""
};
while (result.status.ToLower() != "processed")
{
System.Threading.Thread.Sleep(5000);
result = _sut.GetCollection(collection.id);
}
//assert
Assert.IsNotNull(result.topics.Length > 0);
}