database for a topical guide to this subject see outline of databases an sql select statement and its result a database is an organized collection of data generally stored and accessed electronically from a computer system where databases are more complex they are often developed using formal design and modeling techniques the database management system dbms is the software that interacts with end users applications the database itself to capture and analyze the data and provides facilities to administer the database the sum total of the database the dbms and the associated applications can be referred to as a database system often the term database is also used to loosely refer to any of the dbms the database system or an application associated with the database computer scientists may classify databasemanagement systems according to the database models that they support relational databases became dominant in the 1980s these model data as rows and columns in a series of tables and the vast majority use sql for writing and querying data in the 2000s nonrelational databases became popular referred to as nosql because they use different query languages terminology and overview formally a database refers to a set of related data and the way it is organized access to this data is usually provided by a database management system dbms consisting of an integrated set of computer software that allows users to interact with one or more databases and provides access to all of the data contained in the database although restrictions may exist that limit access to particular data the dbms provides various functions that allow entry storage and retrieval of large quantities of information and provides ways to manage how that information is organized because of the close relationship between them the term database is often used casually to refer to both a database and the dbms used to manipulate it outside the world of professional information technology the term database is often used to refer to any collection of related data such as a spreadsheet or a card index as however size and usage requirements typically necessitate use of a database management system existing dbmss provide various functions that allow management of a database and its data which can be classified into four main functional groups data definition creation modification and removal of definitions that define the organization of the data update insertion modification and deletion of the actual data retrieval providing information in a form directly usable or for further processing by other applications the retrieved data may be made available in a form basically the same as it is stored in the database or in a new form obtained by altering or combining existing data from the database administration registering and monitoring users enforcing data security monitoring performance maintaining data integrity dealing with concurrency control and recovering information that has been corrupted by some event such as an unexpected system failure both a database and its dbms conform to the principles of a particular database model physically database servers are dedicated computers that hold the actual databases and run only the dbms and related software database servers are usually multiprocessor computers with generous memory and raid disk arrays used for stable storage raid is used for recovery of data if any of the disks fail hardware database accelerators connected to one or more servers via a highspeed channel are also used in large volume transaction processing environments dbmss are found at the heart of most database applications dbmss may be built around a custom multitasking kernel with builtin networking support but modern dbmss typically rely on a standard operating system to provide these functions since dbmss comprise a significant market computer and storage vendors often take into account dbms requirements in their own development plans databases and dbmss can be categorized according to the database models that they support such as relational or xml the types of computer they run on from a server cluster to a mobile phone the query languages used to access the database such as sql or xquery and their internal engineering which affects performance scalability resilience and security history the sizes capabilities and performance of databases and their respective dbmss have grown in orders of magnitude these performance increases were enabled by the technology progress in the areas of processors computer memory computer storage and computer networks the development of database technology can be divided into three eras based on data model or structure navigational sqlrelational and postrelational the two main early navigational data models were the hierarchical model and the codasyl model network model the relational model first proposed in 1970 by edgar f codd departed from this tradition by insisting that applications should search for data by content rather than by following links the relational model employs sets of ledgerstyle tables each used for a different type of entity only in the mid1980s did computing hardware become powerful enough to allow the wide deployment of relational systems dbmss plus applications by the early 1990s however relational systems dominated in all largescale data processing applications and as of 2018 object databases were developed in the 1980s to overcome the inconvenience of objectrelational impedance mismatch which led to the coining of the term postrelational and also the development of hybrid objectrelational databases the next generation of postrelational databases in the late 2000s became known as nosql databases introducing fast keyvalue stores and documentoriented databases a competing next generation known as newsql databases attempted new implementations that retained the relationalsql model while aiming to match the high performance of nosql compared to commercially available relational dbmss 1960s navigational dbms further information navigational database basic structure of navigational codasyl database model the introduction of the term database coincided with the availability of directaccess storage disks and drums from the mid1960s onwards the term represented a contrast with the tapebased systems of the past allowing shared interactive use rather than daily batch processing the oxford english dictionary cites a 1962 report by the system development corporation of california as the first to use the term database in a specific technical sense as computers grew in speed and capability a number of generalpurpose database systems emerged by the mid1960s a number of such systems had come into commercial use interest in a standard began to grow and charles bachman author of one such product the integrated data store ids founded the database task group within codasyl the group responsible for the creation and standardization of cobol in 1971 the database task group delivered their standard which generally became known as the codasyl approach and soon a number of commercial products based on this approach entered the market the codasyl approach relied on the manual navigation of a linked data set which was formed into a large network applications could find records by one of three methods use of a primary key known as a calc key typically implemented by hashing navigating relationships called sets from one record to another scanning all the records in a sequential order later systems added btrees to provide alternate access paths many codasyl databases also added a very straightforward query language however in the final tally codasyl was very complex and required significant training and effort to produce useful applications ibm also had their own dbms in 1966 known as information management system ims ims was a development of software written for the apollo program on the system360 ims was generally similar in concept to codasyl but used a strict hierarchy for its model of data navigation instead of codasyls network model both concepts later became known as navigational databases due to the way data was accessed and bachmans 1973 turing award presentation was the programmer as navigator ims is classified 1970s relational dbms edgar codd worked at ibm in san jose california in one of their offshoot offices that was primarily involved in the development of hard disk systems he was unhappy with the navigational model of the codasyl approach notably the lack of a search facility in 1970 he wrote a number of papers that outlined a new approach to database construction that eventually culminated in the groundbreaking a relational model of data for large shared data banks in this paper he described a new system for storing and working with large databases instead of records being stored in some sort of linked list of freeform records as in codasyl codds idea was to use a table of fixedlength records with each table used for a different type of entity a linkedlist system would be very inefficient when storing sparse databases where some of the data for any one record could be left empty the relational model solved this by splitting the data into a series of normalized tables or relations with optional elements being moved out of the main table to where they would take up room only if needed data may be freely inserted deleted and edited in these tables with the dbms doing whatever maintenance needed to present a table view to the applicationuser in the relational model records are linked using virtual keys not stored in the database but defined as needed between the data contained in the records the relational model also allowed the content of the database to evolve without constant rewriting of links and pointers the relational part comes from entities referencing other entities in what is known as onetomany relationship like a traditional hierarchical model and manytomany relationship like a navigational network model thus a relational model can express both hierarchical and navigational models as well as its native tabular model allowing for pure or combined modeling in terms of these three models as the application requires for instance a common use of a database system is to track information about users their name login information various addresses and phone numbers in the navigational approach all of this data would be placed in a single record and unused items would simply not be placed in the database in the relational approach the data would be normalized into a user table an address table and a phone number table for instance records would be created in these optional tables only if the address or phone numbers were actually provided linking the information back together is the key to this system in the relational model some bit of information was used as a key uniquely defining a particular record when information was being collected about a user information stored in the optional tables would be found by searching for this key for instance if the login name of a user is unique addresses and phone numbers for that user would be recorded with the login name as its key this simple relinking of related data back into a single collection is something that traditional computer languages are not designed for just as the navigational approach would require programs to loop in order to collect records the relational approach would require loops to collect information about any one record codds suggestions was a setoriented language that would later spawn the ubiquitous sql using a branch of mathematics known as tuple calculus he demonstrated that such a system could support all the operations of normal databases inserting updating etc as well as providing a simple system for finding and returning sets of data in a single operation codds paper was picked up by two people at berkeley eugene wong and michael stonebraker they started a project known as ingres using funding that had already been allocated for a geographical database project and student programmers to produce code beginning in 1973 ingres delivered its first test products which were generally ready for widespread use in 1979 ingres was similar to system r in a number of ways including the use of a language for data access known as quel over time ingres moved to the emerging sql standard ibm itself did one test implementation of the relational model prtv and a production one business system 12 both now discontinued honeywell wrote mrds for multics and now there are two new implementations alphora dataphor and rel most other dbms implementations usually called relational are actually sql dbmss in 1970 the university of michigan began development of the micro information management system the system remained in production until 1998 integrated approach database machine in the 1970s and 1980s attempts were made to build database systems with integrated hardware and software the underlying philosophy was that such integration would provide higher performance at lower cost examples were ibm system38 the early offering of teradata and the britton lee inc database machine another approach to hardware support for database management was icls cafs accelerator a hardware disk controller with programmable search capabilities in the long term these efforts were generally unsuccessful because specialized database machines could not keep pace with the rapid development and progress of generalpurpose computers thus most database systems nowadays are software systems running on generalpurpose hardware using generalpurpose computer data storage however this idea is still pursued for certain applications by some companies like netezza and oracle exadata late 1970s sql dbms ibm started working on a prototype system loosely based on codds concepts as system r in the early 1970s the first version was ready in 19745 and work then started on multitable systems in which the data could be split so that all of the data for a record some of which is optional did not have to be stored in a single large chunk subsequent multiuser versions were tested by customers in 1978 and 1979 by which time a standardized query language sql had been added codds ideas were establishing themselves as both workable and superior to codasyl pushing ibm to develop a true production version of system r known as sqlds and later database 2 db2 larry ellisons oracle database or more simply oracle started from a different chain based on ibms papers on system r though oracle v1 implementations were completed in 1978 it wasnt until oracle version 2 when ellison beat ibm to market in 1979 stonebraker went on to apply the lessons from ingres to develop a new database postgres which is now known as postgresql postgresql is often used for global mission critical applications the org and info domain name registries use it as their primary data store as do many large companies and financial institutions in sweden codds paper was also read and mimer sql was developed from the mid1970s at uppsala university in 1984 this project was consolidated into an independent enterprise another data model the entityrelationship model emerged in 1976 and gained popularity for database design as it emphasized a more familiar description than the earlier relational model later on entityrelationship constructs were retrofitted as a data modeling construct for the relational model and the difference between the two have become irrelevant 1980s on the desktop the 1980s ushered in the age of desktop computing the new computers empowered their users with spreadsheets like lotus 123 and database software like dbase the dbase product was lightweight and easy for any computer user to understand out of the box c wayne ratliff the creator of dbase stated dbase was different from programs like basic c fortran and cobol in that a lot of the dirty work had already been done the data manipulation is done by dbase instead of by the user so the user can concentrate on what he is doing rather than having to mess with the dirty details of opening reading and closing files and managing space allocation dbase was one of the top selling software titles in the 1980s and early 1990s 1990s objectoriented the 1990s along with a rise in objectoriented programming saw a growth in how data in various databases were handled programmers and designers began to treat the data in their databases as objects that is to say that if a persons data were in a database that persons attributes such as their address phone number and age were now considered to belong to that person instead of being extraneous data this allows for relations between data to be relations to objects and their attributes and not to individual fields the term objectrelational impedance mismatch described the inconvenience of translating between programmed objects and database tables object databases and objectrelational databases attempt to solve this problem by providing an objectoriented language sometimes as extensions to sql that programmers can use as alternative to purely relational sql on the programming side libraries known as objectrelational mappings orms attempt to solve the same problem 2000s nosql and newsql nosql and newsql xml databases are a type of structured documentoriented database that allows querying based on xml document attributes xml databases are mostly used in applications where the data is conveniently viewed as a collection of documents with a structure that can vary from the very flexible to the highly rigid examples include scientific articles patents tax filings and personnel records nosql databases are often very fast do not require fixed table schemas avoid join operations by storing denormalized data and are designed to scale horizontally in recent years there has been a strong demand for massively distributed databases with high partition tolerance but according to the cap theorem it is impossible for a distributed system to simultaneously provide consistency availability and partition tolerance guarantees a distributed system can satisfy any two of these guarantees at the same time but not all three for that reason many nosql databases are using what is called eventual consistency to provide both availability and partition tolerance guarantees with a reduced level of data consistency newsql is a class of modern relational databases that aims to provide the same scalable performance of nosql systems for online transaction processing readwrite workloads while still using sql and maintaining the acid guarantees of a traditional database system use cases databases are used to support internal operations of organizations and to underpin online interactions with customers and suppliers see enterprise software databases are used to hold administrative information and more specialized data such as engineering data or economic models examples include computerized library systems flight reservation systems computerized parts inventory systems and many content management systems that store websites as collections of webpages in a database classification one way to classify databases involves the type of their contents for example bibliographic documenttext statistical or multimedia objects another way is by their application area for example accounting music compositions movies banking manufacturing or insurance a third way is by some technical aspect such as the database structure or interface type this section lists a few of the adjectives used to characterize different kinds of databases an inmemory database is a database that primarily resides in main memory but is typically backedup by nonvolatile computer data storage main memory databases are faster than disk databases and so are often used where response time is critical such as in telecommunications network equipment an active database includes an eventdriven architecture which can respond to conditions both inside and outside the database possible uses include security monitoring alerting statistics gathering and authorization many databases provide active database features in the form of database triggers a cloud database relies on cloud technology both the database and most of its dbms reside remotely in the cloud while its applications are both developed by programmers and later maintained and used by endusers through a web browser and open apis data warehouses archive data from operational databases and often from external sources such as market research firms the warehouse becomes the central source of data for use by managers and other endusers who may not have access to operational data for example sales data might be aggregated to weekly totals and converted from internal product codes to use upcs so that they can be compared with acnielsen data some basic and essential components of data warehousing include extracting analyzing and mining data transforming loading and managing data so as to make them available for further use a deductive database combines logic programming with a relational database a distributed database is one in which both the data and the dbms span multiple computers a documentoriented database is designed for storing retrieving and managing documentoriented or semi structured information documentoriented databases are one of the main categories of nosql databases an embedded database system is a dbms which is tightly integrated with an application software that requires access to stored data in such a way that the dbms is hidden from the applications endusers and requires little or no ongoing maintenance enduser databases consist of data developed by individual endusers examples of these are collections of documents spreadsheets presentations multimedia and other files several products exist to support such databases some of them are much simpler than fullfledged dbmss with more elementary dbms functionality a federated database system comprises several distinct databases each with its own dbms it is handled as a single database by a federated database management system fdbms which transparently integrates multiple autonomous dbmss possibly of different types in which case it would also be a heterogeneous database system and provides them with an integrated conceptual view sometimes the term multidatabase is used as a synonym to federated database though it may refer to a less integrated eg without an fdbms and a managed integrated schema group of databases that cooperate in a single application in this case typically middleware is used for distribution which typically includes an atomic commit protocol acp eg the twophase commit protocol to allow distributed global transactions across the participating databases a graph database is a kind of nosql database that uses graph structures with nodes edges and properties to represent and store information general graph databases that can store any graph are distinct from specialized graph databases such as triplestores and network databases an array dbms is a kind of nosql dbms that allows modeling storage and retrieval of usually large multidimensional arrays such as satellite images and climate simulation output in a hypertext or hypermedia database any word or a piece of text representing an object eg another piece of text an article a picture or a film can be hyperlinked to that object hypertext databases are particularly useful for organizing large amounts of disparate information for example they are useful for organizing online encyclopedias where users can conveniently jump around the text the world wide web is thus a large distributed hypertext database a knowledge base abbreviated kb kb or is a special kind of database for knowledge management providing the means for the computerized collection organization and retrieval of knowledge also a collection of data representing problems with their solutions and related experiences a mobile database can be carried on or synchronized from a mobile computing device operational databases store detailed data about the operations of an organization they typically process relatively high volumes of updates using transactions examples include customer databases that record contact credit and demographic information about a businesss customers personnel databases that hold information such as salary benefits skills data about employees enterprise resource planning systems that record details about product components parts inventory and financial databases that keep track of the organizations money accounting and financial dealings a parallel database seeks to improve performance through parallelization for tasks such as loading data building indexes and evaluating queries the major parallel dbms architectures which are induced by the underlying hardware architecture are shared memory architecture where multiple processors share the main memory space as well as other data storage shared disk architecture where each processing unit typically consisting of multiple processors has its own main memory but all units share the other storage shared nothing architecture where each processing unit has its own main memory and other storage probabilistic databases employ fuzzy logic to draw inferences from imprecise data realtime databases process transactions fast enough for the result to come back and be acted on right away a spatial database can store the data with multidimensional features the queries on such data include locationbased queries like where is the closest hotel in my area a temporal database has builtin time aspects for example a temporal data model and a temporal version of sql more specifically the temporal aspects usually include validtime and transactiontime a terminologyoriented database builds upon an objectoriented database often customized for a specific field an unstructured data database is intended to store in a manageable and protected way diverse objects that do not fit naturally and conveniently in common databases it may include email messages documents journals multimedia objects etc the name may be misleading since some objects can be highly structured however the entire possible object collection does not fit into a predefined structured framework most established dbmss now support unstructured data in various ways and new dedicated dbmss are emerging database interaction database management system connolly and begg define database management system dbms as a software system that enables users to define create maintain and control access to the database the dbms acronym is sometime extended to indicated the underlying database model with rdbms for relational oodbms or ordbms for the object orientated model and ordbms for objectrelational other extensions can indicate some other characteristic such as ddbms for a distributed database management systems the functionality provided by a dbms can vary enormously the core functionality is the storage retrieval and update of data codd proposed the following functions and services a fullyfledged general purpose dbms should provide data storage retrieval and update user accessible catalog or data dictionary describing the metadata support for transactions and concurrency facilities for recovering the database should it become damaged support for authorization of access and update of data access support from remote locations enforcing constraints to ensure data in the database abides by certain rules it is also generally to be expected the dbms will provide a set of utilities for such purposes as may be necessary to administer the database effectively including import export monitoring defragmentation and analysis utilities the core part of the dbms interacting between the database and the application interface sometimes referred to as the database engine often dbmss will have configuration parameters that can be statically and dynamically tuned for example the maximum amount of main memory on a server the database can use the trend is to minimise the amount of manual configuration and for cases such as embedded databases the need to target zeroadministration is paramount the large major enterprise dbmss have tended to increase in size and functionality and can have involved thousands of human years of development effort through their lifetime early multiuser dbms typically only allowed for the application to reside on the same computer with access via terminals or terminal emulation software the clientserver architecture was a development where the application resided on a client desktop and the database on a server allowing the processing to be distributed this evolved into a multitier architecture incorporating application servers and web servers with the end user interface via a web browser with the database only directly connected to the adjacent tier a generalpurpose dbms will provide public application programming interfaces api and optionally a processor for database languages such as sql to allow applications to be written to interact with the database a special purpose dbms may use a private api and be specifically customised and linked to a single application for example an email system performing many of the functions of a generalpurpose dbms such as message insertion message deletion attachment handling blocklist lookup associating messages an email address and so forth however these functions are limited to what is required to handle email application external interaction with the database will be via an application program that interfaces with the dbms this can range from a database tool that allows users to execute sql queries textually or graphically to a web site that happens to use a database to store and search information application program interface a programmer will code interactions to the database sometimes referred to as a datasource via an application program interface api or via a database language the particular api or language chosen will need to be supported by dbms possible indirectly via a preprocessor or a bridging api some apis aim to be database independent odbc being a commonly known example other common apis include jdbc and adonet database languages database languages are specialpurpose languages which allow one or more of the following tasks sometimes distinguished as sublanguages data control language dcl controls access to data data definition language ddl defines data types such as creating altering or dropping and the relationships among them data manipulation language dml performs tasks such as inserting updating or deleting data occurrences data query language dql allows searching for information and computing derived information database languages are specific to a particular data model notable examples include sql combines the roles of data definition data manipulation and query in a single language it was one of the first commercial languages for the relational model although it departs in some respects from the relational model as described by codd for example the rows and columns of a table can be ordered sql became a standard of the american national standards institute ansi in 1986 and of the international organization for standardization iso in 1987 the standards have been regularly enhanced since and is supported with varying degrees of conformance by all mainstream commercial relational dbmss oql is an object model language standard from the object data management group it has influenced the design of some of the newer query languages like jdoql and ejb ql xquery is a standard xml query language implemented by xml database systems such as marklogic and exist by relational databases with xml capability such as oracle and db2 and also by inmemory xml processors such as saxon sqlxml combines xquery with sql a database language may also incorporate features like dbmsspecific configuration and storage engine management computations to modify query results like counting summing averaging sorting grouping and crossreferencing constraint enforcement eg in an automotive database only allowing one engine type per car application programming interface version of the query language for programmer convenience storage computer data storage and database engine database storage is the container of the physical materialization of a database it comprises the internal physical level in the database architecture it also contains all the information needed eg metadata data about the data and internal data structures to reconstruct the conceptual level and external level from the internal level when needed putting data into permanent storage is generally the responsibility of the database engine aka storage engine though typically accessed by a dbms through the underlying operating system and often using the operating systems file systems as intermediates for storage layout storage properties and configuration setting are extremely important for the efficient operation of the dbms and thus are closely maintained by database administrators a dbms while in operation always has its database residing in several types of storage eg memory and external storage the database data and the additional needed information possibly in very large amounts are coded into bits data typically reside in the storage in structures that look completely different from the way the data look in the conceptual and external levels but in ways that attempt to optimize the best possible these levels reconstruction when needed by users and programs as well as for computing additional types of needed information from the data eg when querying the database some dbmss support specifying which character encoding was used to store data so multiple encodings can be used in the same database various lowlevel database storage structures are used by the storage engine to serialize the data model so it can be written to the medium of choice techniques such as indexing may be used to improve performance conventional storage is roworiented but there are also columnoriented and correlation databases materialized views materialized view often storage redundancy is employed to increase performance a common example is storing materialized views which consist of frequently needed external views or query results storing such views saves the expensive computing of them each time they are needed the downsides of materialized views are the overhead incurred when updating them to keep them synchronized with their original updated database data and the cost of storage redundancy replication database replication occasionally a database employs storage redundancy by database objects replication with one or more copies to increase data availability both to improve performance of simultaneous multiple enduser accesses to a same database object and to provide resiliency in a case of partial failure of a distributed database updates of a replicated object need to be synchronized across the object copies in many cases the entire database is replicated security this article appears to contradict the article database security please see discussion on the linked talk page march 2013 learn how and when to remove this template message database security database security deals with all various aspects of protecting the database content its owners and its users it ranges from protection from intentional unauthorized database uses to unintentional database accesses by unauthorized entities eg a person or a computer program database access control deals with controlling who a person or a certain computer program is allowed to access what information in the database the information may comprise specific database objects eg record types specific records data structures certain computations over certain objects eg query types or specific queries or using specific access paths to the former eg using specific indexes or other data structures to access information database access controls are set by special authorized by the database owner personnel that uses dedicated protected security dbms interfaces this may be managed directly on an individual basis or by the assignment of individuals and privileges to groups or in the most elaborate models through the assignment of individuals and groups to roles which are then granted entitlements data security prevents unauthorized users from viewing or updating the database using passwords users are allowed access to the entire database or subsets of it called subschemas for example an employee database can contain all the data about an individual employee but one group of users may be authorized to view only payroll data while others are allowed access to only work history and medical data if the dbms provides a way to interactively enter and update the database as well as interrogate it this capability allows for managing personal databases data security in general deals with protecting specific chunks of data both physically ie from corruption or destruction or removal eg see physical security or the interpretation of them or parts of them to meaningful information eg by looking at the strings of bits that they comprise concluding specific valid creditcard numbers eg see data encryption change and access logging records who accessed which attributes what was changed and when it was changed logging services allow for a forensic database audit later by keeping a record of access occurrences and changes sometimes applicationlevel code is used to record changes rather than leaving this to the database monitoring can be set up to attempt to detect security breaches transactions and concurrency further information concurrency control database transactions can be used to introduce some level of fault tolerance and data integrity after recovery from a crash a database transaction is a unit of work typically encapsulating a number of operations over a database eg reading a database object writing acquiring lock etc an abstraction supported in database and also other systems each transaction has well defined boundaries in terms of which programcode executions are included in that transaction determined by the transactions programmer via special transaction commands the acronym acid describes some ideal properties of a database transaction atomicity consistency isolation and durability migration see also data migration database migration a database built with one dbms is not portable to another dbms ie the other dbms cannot run it however in some situations it is desirable to move migrate a database from one dbms to another the reasons are primarily economical different dbmss may have different total costs of ownership or tcos functional and operational different dbmss may have different capabilities the migration involves the databases transformation from one dbms type to another the transformation should maintain if possible the database related application ie all related application programs intact thus the databases conceptual and external architectural levels should be maintained in the transformation it may be desired that also some aspects of the architecture internal level are maintained a complex or large database migration may be a complicated and costly onetime project by itself which should be factored into the decision to migrate this in spite of the fact that tools may exist to help migration between specific dbmss typically a dbms vendor provides tools to help importing databases from other popular dbmss building maintaining and tuning database tuning after designing a database for an application the next stage is building the database typically an appropriate generalpurpose dbms can be selected to be used for this purpose a dbms provides the needed user interfaces to be used by database administrators to define the needed applications data structures within the dbmss respective data model other user interfaces are used to select needed dbms parameters like security related storage allocation parameters etc when the database is ready all its data structures and other needed components are defined it is typically populated with initial applications data database initialization which is typically a distinct project in many cases using specialized dbms interfaces that support bulk insertion before making it operational in some cases the database becomes operational while empty of application data and data are accumulated during its operation after the database is created initialised and populated it needs to be maintained various database parameters may need changing and the database may need to be tuned tuning for better performance applications data structures may be changed or added new related application programs may be written to add to the applications functionality etc backup and restore backup sometimes it is desired to bring a database back to a previous state for many reasons eg cases when the database is found corrupted due to a software error or if it has been updated with erroneous data to achieve this a backup operation is done occasionally or continuously where each desired database state ie the values of its data and their embedding in databases data structures is kept within dedicated backup files many techniques exist to do this effectively when this state is needed ie when it is decided by a database administrator to bring the database back to this state eg by specifying this state by a desired point in time when the database was in this state these files are used to restore that state static analysis static analysis techniques for software verification can be applied also in the scenario of query languages in particular the abstract interpretation framework has been extended to the field of query languages for relational databases as a way to support sound approximation techniques the semantics of query languages can be tuned according to suitable abstractions of the concrete domain of data the abstraction of relational database system has many interesting applications in particular for security purposes such as fine grained access control watermarking etc miscellaneous features other dbms features might include database logs this helps in keeping a history of the executed functions graphics component for producing graphs and charts especially in a data warehouse system query optimizer performs query optimization on every query to choose an efficient query plan a partial order tree of operations to be executed to compute the query result may be specific to a particular storage engine tools or hooks for database design application programming application program maintenance database performance analysis and monitoring database configuration monitoring dbms hardware configuration a dbms and related database may span computers networks and storage units and related database mapping especially for a distributed dbms storage allocation and database layout monitoring storage migration etc increasingly there are calls for a single system that incorporates all of these core functionalities into the same build test and deployment framework for database management and source control borrowing from other developments in the software industry some market such offerings as devops for database design and modeling database design the first task of a database designer is to produce a conceptual data model that reflects the structure of the information to be held in the database a common approach to this is to develop an entityrelationship model often with the aid of drawing tools another popular approach is the unified modeling language a successful data model will accurately reflect the possible state of the external world being modeled for example if people can have more than one phone number it will allow this information to be captured designing a good conceptual data model requires a good understanding of the application domain it typically involves asking deep questions about the things of interest to an organization like can a customer also be a supplier or if a product is sold with two different forms of packaging are those the same product or different products or if a plane flies from new york to dubai via frankfurt is that one flight or two or maybe even three the answers to these questions establish definitions of the terminology used for entities customers products flights flight segments and their relationships and attributes producing the conceptual data model sometimes involves input from business processes or the analysis of workflow in the organization this can help to establish what information is needed in the database and what can be left out for example it can help when deciding whether the database needs to hold historic data as well as current data having produced a conceptual data model that users are happy with the next stage is to translate this into a schema that implements the relevant data structures within the database this process is often called logical database design and the output is a logical data model expressed in the form of a schema whereas the conceptual data model is in theory at least independent of the choice of database technology the logical data model will be expressed in terms of a particular database model supported by the chosen dbms the terms data model and database model are often used interchangeably but in this article we use data model for the design of a specific database and database model for the modeling notation used to express that design the most popular database model for generalpurpose databases is the relational model or more precisely the relational model as represented by the sql language the process of creating a logical database design using this model uses a methodical approach known as normalization the goal of normalization is to ensure that each elementary fact is only recorded in one place so that insertions updates and deletions automatically maintain consistency the final stage of database design is to make the decisions that affect performance scalability recovery security and the like which depend on the particular dbms this is often called physical database design and the output is the physical data model a key goal during this stage is data independence meaning that the decisions made for performance optimization purposes should be invisible to endusers and applications there are two types of data independence physical data independence and logical data independence physical design is driven mainly by performance requirements and requires a good knowledge of the expected workload and access patterns and a deep understanding of the features offered by the chosen dbms another aspect of physical database design is security it involves both defining access control to database objects as well as defining security levels and methods for the data itself models database model collage of five types of database models a database model is a type of data model that determines the logical structure of a database and fundamentally determines in which manner data can be stored organized and manipulated the most popular example of a database model is the relational model or the sql approximation of relational which uses a tablebased format common logical data models for databases include navigational databases hierarchical database model network model graph database relational model entityrelationship model enhanced entityrelationship model object model document model entityattributevalue model star schema an objectrelational database combines the two related structures physical data models include inverted index flat file other models include associative model multidimensional model array model multivalue model specialized models are optimized for particular types of data xml database semantic model content store event store time series model external conceptual and internal views traditional view of data a database management system provides three views of the database data the external level defines how each group of endusers sees the organization of data in the database a single database can have any number of views at the external level the conceptual level unifies the various external views into a compatible global view it provides the synthesis of all the external views it is out of the scope of the various database endusers and is rather of interest to database application developers and database administrators the internal level or physical level is the internal organization of data inside a dbms it is concerned with cost performance scalability and other operational matters it deals with storage layout of the data using storage structures such as indexes to enhance performance occasionally it stores data of individual views materialized views computed from generic data if performance justification exists for such redundancy it balances all the external views performance requirements possibly conflicting in an attempt to optimize overall performance across all activities while there is typically only one conceptual or logical and physical or internal view of the data there can be any number of different external views this allows users to see database information in a more businessrelated way rather than from a technical processing viewpoint for example a financial department of a company needs the payment details of all employees as part of the companys expenses but does not need details about employees that are the interest of the human resources department thus different departments need different views of the companys database the threelevel database architecture relates to the concept of data independence which was one of the major initial driving forces of the relational model the idea is that changes made at a certain level do not affect the view at a higher level for example changes in the internal level do not affect application programs written using conceptual level interfaces which reduces the impact of making physical changes to improve performance the conceptual view provides a level of indirection between internal and external on one hand it provides a common view of the database independent of different external view structures and on the other hand it abstracts away details of how the data are stored or managed internal level in principle every level and even every external view can be presented by a different data model in practice usually a given dbms uses the same data model for both the external and the conceptual levels eg relational model the internal level which is hidden inside the dbms and depends on its implementation requires a different level of detail and uses its own types of data structure types separating the external conceptual and internal levels was a major feature of the relational database model implementations that dominate 21st century databases research database technology has been an active research topic since the 1960s both in academia and in the research and development groups of companies for example ibm research research activity includes theory and development of prototypes notable research topics have included models the atomic transaction concept and related concurrency control techniques query languages and query optimization methods raid and more the database research area has several dedicated academic journals for example acm transactions on database systemstods data and knowledge engineeringdke and annual conferences eg acm sigmod acm pods vldb ieee icde for a topical guide to this subject see outline of databases book databases comparison of database tools comparison of object database management systems comparison of objectrelational database management systems comparison of relational database management systems data hierarchy data bank data store database theory database testing databasecentric architecture journal of database management questionfocused dataset notes bachman charles w 1973 the programmer as navigator communications of the acm 16 11 653658 doi101145355611362534 subscription required help beynondavies paul 2003 database systems 3rd ed palgrave macmillan isbn9781403916013 chapple mike 2005 sql fundamentals databases aboutcom archived from the original on 22 february 2009 retrieved 28 january 2009 childs david l 1968a description of a settheoretic data structure pdf concomp research in conversational use of computers project technical report 3 university of michigan childs david l 1968b feasibility of a settheoretic data structure a general structure based on a reconstituted definition pdf concomp research in conversational use of computers project technical report 6 university of michigan chong raul f wang xiaomei dang michael snow dwaine r 2007 introduction to db2 understanding db2 learning visually with examples 2nd ed isbn9780131580183 retrieved 17 march 2013 codd edgar f 1970 a relational model of data for large shared data banks pdf communications of the acm 13 6 377387 doi101145362384362685 connolly thomas m begg carolyn e 2014 database systems a practical approach to design implementation and management 6th ed pearson isbn9781292061184 date c j 2003 an introduction to database systems 8th ed pearson isbn9780321197849 halder raju cortesi agostino 2011 abstract interpretation of database query languages pdf computer languages systems structures elsevier 38 2 123157 doi101016jcl201110004 issn14778424 hershey william easthope carol 1972 a set theoretic data structure and retrieval language spring joint computer conference may 1972 acm sigir forum 7 4 pp4555 doi10114510954951095500 nelson anne fulcher nelson william harris morehead 2001 building electronic commerce with web database constructions prentice hall isbn9780201741308 north ken 10 march 2010 sets data models and data independence dr dobbs archived from the original on 24 october 2010 tsitchizris dionysios c lochovsky fred h 1982 data models prenticehall isbn9780131964280 ullman jeffrey widom jennifer 1997 a first course in database systems prenticehall isbn0138613370 wagner michael 2010 sqlxml2006 evaluierung der standardkonformitt ausgewhlter datenbanksysteme diplomica verlag isbn9783836696098 ling liu and tamer m zsu eds 2009 encyclopedia of database systems 4100 p60 illus isbn9780387496160 gray j and reuter a transaction processing concepts and techniques 1st edition morgan kaufmann publishers 1992 kroenke david m and david j auer database concepts 3rd ed new york prentice 2007 raghu ramakrishnan and johannes gehrke database management systems abraham silberschatz henry f korth s sudarshan database system concepts lightstone s teorey t nadeau t 2007 physical database design the database professionals guide to exploiting indexes views storage and more morgan kaufmann press isbn0123693896 teorey t lightstone s and nadeau t database modeling design logical design 4th edition morgan kaufmann press 2005 isbn0126853525 databaseat wikipedias sister projects definitions from wiktionary media from wikimedia commons news from wikinews quotations from wikiquote texts from wikisource textbooks from wikibooks resources from wikiversity db file extension information about files with the db extension vtedatabasemain requirements theory models database management system machine server application connection datasource dsn administrator lock types tools languages data definition data manipulation query information retrieval security activity monitoring audit forensics negative database design entities and relationships and enhanced notation normalization refactoring programming abstraction layer objectrelational mapping management virtualization tuning caching migration preservation integrity see also databasecentric architecture intelligent database twophase locking locks with ordered sharing load file publishing halloween problem log shipping book category wikiproject vtedatabase management systemstypes objectoriented comparison relational comparison documentoriented graph nosql newsql concepts database acid armstrongs axioms cap theorem crud null candidate key foreign key superkey surrogate key unique key objects relation table column row view transaction transaction log trigger index stored procedure cursor partition components concurrency control data dictionary jdbc xqj odbc query language query optimizer query plan functions administration and automation query optimization replication related topics database models database normalization database storage distributed database federated database system referential integrity relational algebra relational calculus relational database relational model objectrelational database transaction processing vtedatabase modelscommon models flat hierarchical dimensional network relational entityrelationship enhanced graph objectoriented entityattributevalue other models associative correlational multidimensional array semantic star schema xml database implementations flat file columnoriented documentoriented objectrelational deductive temporal xml data store keyvalue store triplestore vtedata warehousecreating the data warehouseconcepts database dimension dimensional modeling fact olap star schema snowflake schema reverse star schema aggregate variants anchor modeling columnoriented dbms data vault modeling holap molap rolap operational data store elements data dictionarymetadata data mart sixth normal form surrogate key fact fact table earlyarriving fact measure dimension dimension table degenerate slowly changing filling extracttransformload etl extract transform load using the data warehouseconcepts business intelligence dashboard data mining decision support system dss olap cube data warehouse automation languages data mining extensions dmx multidimensional expressions mdx xml for analysis xmla tools business intelligence software reporting software spreadsheet relatedpeople bill inmon ralph kimball products comparison of olap servers data warehousing products and their producers authority control gnd 41132762 lccn sh99001528 nara 10634624 ndl 00865521 