Data-intensive computing this approach is considered suitable for data-intensive computing and problems which are “embarrassingly parallel”, i.e. where it is relatively easy to separate the problem into a number of parallel tasks and there is no dependency or communication required between the tasks other than overall management of the tasks. These types of data processing problems are inherently adaptable to various forms of distributed computing including clusters, data grids, and cloud computing. Several common characteristics of data-intensive computing systems distinguish them from other forms of computing: A variety of system architectures have been implemented for data-intensive computing and large-scale data analysis applications
Data-intensive computing framework for data-intensive computing. Pig was developed at Yahoo! to provide a specific language notation for data analysis applications and to improve programmer productivity and reduce development cycles when using the Hadoop MapReduce environment. Pig programs are automatically translated into sequences of MapReduce programs if needed in the execution environment. Pig provides capabilities in the language for loading, storing, filtering, grouping, de-duplication, ordering, sorting, aggregation, and joining operations on the data. HPCC (High-Performance Computing Cluster) was developed and implemented by LexisNexis Risk Solutions. The development of this computing platform began in 1999 and applications were in production by late 2000.