Refine
Document Type
- Doctoral Thesis (3)
- Book (1)
Has Fulltext
- yes (4)
Is part of the Bibliography
- no (4)
Keywords
- Qualitätssicherung (4) (remove)
In the last decade, crowdsourcing has proved its ability to address large scale data collection tasks, such as labeling large data sets, at a low cost and in a short time. However, the performance and behavior variability between workers as well as the variability in task designs and contents, induce an unevenness in the quality of the produced contributions and, thus, in the final output quality. In order to maintain the effectiveness of crowdsourcing, it is crucial to control the quality of the contributions. Furthermore, maintaining the efficiency of crowdsourcing requires the time and cost overhead related to the quality control to be at its lowest. While effective, current quality control techniques such as contribution aggregation, worker selection, context-specific reputation systems, and multi-step workflows, suffer from fairly high time and budget overheads and from their dependency on prior knowledge about individual workers.
In this thesis, we address this challenge by leveraging the similarity between completed and incoming tasks as well as the correlation between the worker declarative profiles and their performance in previous tasks in order to perform an efficient task-aware worker selection. To this end, we propose CAWS (Context AwareWorker Selection) method which operates in two phases; in an offline phase, completed tasks are clustered into homogeneous groups for each of which the correlation with the workers declarative profile is learned. Then, in the online phase, incoming tasks are matched to one of the existing clusters and the correspondent, previously inferred profile model is used to select the most reliable online workers for the given task. Using declarative profiles helps eliminate any probing process, which reduces the time and the budget while maintaining the crowdsourcing quality. Furthermore, the set of completed tasks, when compared to a probing task split, provides a larger corpus from which a more precise profile model can be learned. This translates to a better selection quality, especially for harder tasks.
In order to evaluate CAWS, we introduce CrowdED (Crowdsourcing Evaluation Dataset), a rich dataset to evaluate quality control methods and quality-driven task vectorization and clustering. The generation of CrowdED relies on a constrained sampling approach that allows to produce a task corpus which respects both, the budget and type constraints. Beside helping in evaluating CAWS, and through its generality and richness, CrowdED helps in plugging the benchmarking gap present in the crowdsourcing quality control community.
Using CrowdED, we evaluate the performance of CAWS in terms of the quality of the worker selection and in terms of the achieved time and budget reduction. Results shows the following: first, automatic grouping is able to achieve a learning quality similar to job-based grouping. And second, CAWS is able to outperform the state-of-the-art profile-based worker selection when it comes to quality. This is especially true when strong budget and time constraints are present on the requester side.
Finally, we complement our work by a software contribution consisting of an open source framework called CREX (CReate Enrich eXtend). CREX allows the creation, the extension and the enrichment of crowdsourcing datasets. It provides the tools to vectorize, cluster and sample a task corpus to produce constrained task sets and to automatically generate custom crowdsourcing campaign sites.
Software has become an important part of our life. Therefore, the number of different applications scenarios and user requirements of software systems grows rapidly. To satisfy these requirements, software vendors build configurable software systems that can be tailored to diverse needs without rebuilding them from scratch, which reduces costs and development time.
Despite considerable advances in software engineering, which allow building high-quality configurable software systems, some challenges remain. One of these challenges is the feature interaction problem that arises when parts (features), from which a configurable system is composed, interact in unexpected ways, and inadvertently change the behavior or quality attributes (such as performance) of the system.
The goal of this dissertation is to systematically study the nature of feature interactions, their causes, their influence on performance of configurable systems, and, based on empirical results, suggest ways of improving techniques for detecting and predicting feature interactions.
More specifically, we compared and evaluated different strategies for the analysis of configurable software systems. The results of our evaluation complement empirical data from previous work about how different analysis strategies for configurable software systems compare with respect to different aspects, such as performance. These results shall be used to develop effective and scalable techniques and tools for analysis of configurable software including feature-interaction detection and prediction techniques and tools.
Technically, we used a machine-learning technique to quantify the influence of feature interactions on performance of real-world configurable systems. We studied the characteristics of interactions that have the largest influence on performance and found that interactions among few features have higher influence than interactions among many features. With a growing number of interacting features, the influence of the corresponding interactions decreases consistently. This implies that interactions involving multiple features can be ignored in practice because of their marginal influence on performance. We also investigated the causes of the interactions and were able to identify several patterns that link these interactions to the architecture of the systems: For example, we found that if a data processing system consisted of multiple features that processed the same data in sequence then these features interacted. The identified patterns can help to anticipate performance interactions already at an early development stage when a system’s architecture is designed.
Furthermore, considering that control-flow interactions (observable at the level of control flow among features) are easier to detect than performance interactions (externally observable through measuring performance of different combinations of features), we conducted a case study on two configurable systems. In this case study, we investigated a possible relation among control-flow feature interactions and performance feature interactions. We also discussed how this relation can be exploited by interaction detection and performance prediction techniques to make them more time efficient and precise. Our case study on two real-world configurable systems revealed that a relation indeed exists, and we were able to show how it can be used to reduce the search space of possibly existing performance interactions. The study can serve as a blueprint for further studies that can rely on our conceptual framework for investigating relations among external and internal interactions.
Overall, the contribution of this dissertation consists of scientific and technical insights, practical tool implementations, empirical evaluations, and case studies that advance the current state of research in the area of feature interactions in configurable software systems. In particular, we provide insights into the causes of feature interactions and their influence on performance of real-world configurable systems (e.g., interaction patterns, decreasing influence of interactions with growing number of involved features). Our results also suggest ways of improving techniques for detecting and predicting feature interactions (e.g., ignoring interactions among multiple features, reducing the search space based on relations among interactions).
VORWORT
Längst wird auch im Zusammenhang mit medizinischen Behandlungen eine Diskussion über Maßnahmen der Qualitätssicherung geführt. Diese geht von der medizinischen Wissenschaft aus, erfasst aber auch die Rechtswissenschaft und dabei auch das Recht der gesetzlichen Krankenversicherung. Der vorliegende Band untersucht ausgehend von medizinischen und rechtlichen Vorgaben die Frage, welche Auswirkungen Maßnahmen der Qualitätssicherung auch auf das Recht der gesetzlichen Krankenversicherung haben. Mit der Einbeziehung von Medizinern und Juristen wird die nötige Ausgangsbasis für die Untersuchung der einschlägigen Vorschriften zur Qualitätssicherung im Krankenversicherungsrecht hergestellt.
Der vorliegende Band geht auf die Referate im Rahmen der fünften „deutsch-österreichischen Sozialrechtsgespräche" zurück, welche am 30. und 31. Jänner 2003 an der Universität Linz als Kooperation des Instituts für Arbeitsrecht und Sozialrecht der Universität Linz, des Lehrstuhls für Staats-und Verwaltungsrecht, insbesondere Sozialrecht der Universität Passau und der Oberösterreichischen Gebietskrankenkasse stattgefunden haben. Die Herausgeber schulden der Oberösterreichischen Gebietskrankenkasse Dank für die finanzielle und organisatorische Unterstützung der Tagung.
INHALTSVERZEICHNIS
Georg Entmayr
Rechtliche Überlegungen zur Qualitätssicherung im niedergelassenen Bereich nach §343 Abs5 ASVG 1
Otfried Seewald
Definition und Ziele der medizinischen Qualitätssicherung in Deutschland 11
Karl Stöger
Qualitätssicherung an der Schnittstelle zwischen Medizin und Recht 35
Roland Benkowitsch
Leitlinien und medizinische Standards als Instrument der Qualitätssicherung in Deutschland 65
Gerhard Aigner
Leitlinien und medizinische Standards als Instrumente der Qualitätssicherung in Österreich 81
Helmut Platzer
Qualitätskontrolle aus kassenrechtlicber Sicht in Deutschland 93
Rudolf Mosler
Qualitätskontrolle (insbesondere auch aus kassenrechtlicher Sicht) in Österreich 105
Diese Arbeit präsentiert eine neue Methode zur Sicherheitsanalyse von Software im Bereich der Manipulationsprüfung und der Einhaltung von Informationsflüssen zwischen verschiedenen Sicherheitsniveaus. Program-Slicing und Constraint-Solving sind eigenständige Verfahren, die sowohl zur Abhängigkeitsbestimmung als auch zur Berechnung arithmetischer Eigenschaften verwendet werden. Die erstmalige Kombination dieser beiden Verfahren mittels Pfadbedingungen liefert nicht nur binäre Abhängigkeitsinformationen wie Slicing, sondern exakte notwendige Bedingungen über die Informationsflüsse zwischen zwei Programmpunkten. Neben der Definition der Grundlagen von Abhängigkeitsgraphen und einfachen Pfadbedingungen werden neue Erweiterungen für kontextsensitive interprozedurale Pfadbedingungen gezeigt und die Integration von domänenspezifischen Verfahren für Arrayfelder und abstrakten Datentypen demonstriert. Der Schwerpunkt der Arbeit liegt in der Realisierung von Pfadbedingungen für echte Programme in echten Programmiersprachen. Hierfür werden Verfahren vorgeschlagen, realisiert und empirisch untersucht, wie Pfadbedingungen für große Programme skalieren. Die zum Einsatz kommenden Techniken sind u.a. Intervallanalyse und Binäre Entscheidungsgraphen, mit denen die generelle exponentielle Komplexität von Pfadbedingungen beherrschbar wird. Fallstudien für den Einsatz von Pfadbedingungen und die empirische Untersuchung mehrerer Verfahren zur Intervallanalyse zeigen, dass Pfadbedingungen für die praktische Programmanalyse und das Programmverstehen geeignet und empfehlenswert sind.