{"id":1174,"date":"2017-10-26T12:04:09","date_gmt":"2017-10-26T09:04:09","guid":{"rendered":"https:\/\/community.virtono.com\/?p=1174"},"modified":"2020-06-10T16:45:49","modified_gmt":"2020-06-10T13:45:49","slug":"yahoos-big-data-engine-will-be-open-source-project","status":"publish","type":"post","link":"https:\/\/www.virtono.com\/community\/internet-and-technology-news\/yahoos-big-data-engine-will-be-open-source-project\/","title":{"rendered":"Yahoo&#8217;s Big Data Engine will be open source project"},"content":{"rendered":"<header class=\"cluster-header\">Since 2016,\u00a0Yahoo\u00a0belongs to the Verizon Group, and to Oath, the Verizon Digital Network.\u00a0Oath now puts Yahoos&#8217;s Big Data Engine Vespa under Java into an open source license.<\/p>\n<\/header>\n<div class=\"formatted\">\n<p id=\"gpar1\">According to the announcement of\u00a0the Java project\u00a0Vespa\u00a0by Yahoo, it is intended to make it easier for software administrators to create applications that filter out real-time results from a large amount of data in order to present them to the user.\u00a0While Hadoop and Storm, according to the announcement, help to process the data, the last step to deliver them is still a problem.\u00a0Vespa wants to close this gap and is larger with regard to the code lines than any previously released open source project of Yahoo.<\/p>\n<p id=\"gpar2\">The software is used at Oath among others for Yahoo.com, Yahoo News or Flickr.\u00a0It should deliver 9,000 times per second content and ads, with latencies in the tenths of a millisecond range.\u00a0For Flickr, for example, Vespa handles hundreds of queries per second and roars a few billions of images.\u00a0Via Yahoo Gemini, Vespa handles around three billion native ads per day.<\/p>\n<p id=\"gpar3\">The data and their calculations are distributed to Vespa on many machines, the software dispensing with a master that could become a bottle neck.\u00a0Unlike conventional applications, Vespa does not drag the data into a stateless layer to process it, but it does the calculations on the data.\u00a0For this purpose, the software manages clusters with many nodes, which redundantly distribute data in the background, provides new capacities, implements distributed low latency query and processing algorithms, takes care of the consistency of distributed data, and much more.<\/p>\n<p id=\"gpar4\">Vespa allows application developers to feed data and models of any size into the operating system and perform the final calculations at the desired time.\u00a0According to Oath, this improves the user experience at lower costs and allows more complex responses because Vespa waives pre-computing responses to inquiries.\u00a0Developers were more interactive by navigating in real-time and interacting with complex computations instead of starting offline jobs to check the results later.\u00a0The\u00a0code for Vespa is available on Github\u00a0under Apache license 2.0.<\/p>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>Since 2016,\u00a0Yahoo\u00a0belongs to the Verizon Group, and to Oath, the Verizon Digital Network.\u00a0Oath now puts Yahoos&#8217;s Big Data Engine Vespa under Java into an open source license. According to the announcement of\u00a0the Java project\u00a0Vespa\u00a0by Yahoo, it is intended to make it easier for software administrators to create applications that filter<\/p>\n","protected":false},"author":4,"featured_media":1175,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[25],"tags":[],"class_list":["post-1174","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-internet-and-technology-news"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"https:\/\/i0.wp.com\/www.virtono.com\/community\/wp-content\/uploads\/2017\/10\/Vespa-Logo.jpg?fit=720%2C470&ssl=1","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p7ISfL-iW","jetpack_likes_enabled":true,"jetpack-related-posts":[{"id":203,"url":"https:\/\/www.virtono.com\/community\/news-announcements\/softbank-acquires-arm-holdings-for-32-billion\/","url_meta":{"origin":1174,"position":0},"title":"SoftBank Acquires ARM Holdings for $32 billion","author":"Daniel Draga","date":"July 27, 2016","format":false,"excerpt":"ARM, bet you\u2019ve heard that name before, is a multinational semiconductor and software design company hailing out of Cambridge, England. And the name that you might be familiar with is ARM architecture, with nearly every smartphone, tablet, e-reader and smartwatch using it, ARM architecture is an integral part of the\u2026","rel":"","context":"In &quot;Announcements&quot;","block_context":{"text":"Announcements","link":"https:\/\/www.virtono.com\/community\/category\/news-announcements\/"},"img":{"alt_text":"arm 1","src":"https:\/\/i0.wp.com\/community.virtono.com\/wp-content\/uploads\/2016\/07\/arm-1-300x102.jpg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/community.virtono.com\/wp-content\/uploads\/2016\/07\/arm-1-300x102.jpg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/community.virtono.com\/wp-content\/uploads\/2016\/07\/arm-1-300x102.jpg?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/community.virtono.com\/wp-content\/uploads\/2016\/07\/arm-1-300x102.jpg?resize=700%2C400&ssl=1 2x"},"classes":[]},{"id":3856,"url":"https:\/\/www.virtono.com\/community\/tutorial-how-to\/how-to-install-apache-maven-on-ubuntu\/","url_meta":{"origin":1174,"position":1},"title":"How to Install Apache Maven on Ubuntu","author":"George B.","date":"September 9, 2023","format":false,"excerpt":"This tutorial aims to provide a detailed, step-by-step guide on how to install Apache Maven on Ubuntu, a popular open-source operating system. In the realm of software development, Apache Maven stands as a robust project management tool, widely embraced for its ability to handle project builds, reporting, and documentation from\u2026","rel":"","context":"In &quot;Tutorials&quot;","block_context":{"text":"Tutorials","link":"https:\/\/www.virtono.com\/community\/category\/tutorial-how-to\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.virtono.com\/community\/wp-content\/uploads\/2023\/09\/How-to-Install-Apache-Maven-on-Ubuntu-22.png?fit=360%2C240&ssl=1&resize=350%2C200","width":350,"height":200},"classes":[]},{"id":3801,"url":"https:\/\/www.virtono.com\/community\/tutorial-how-to\/how-to-install-apache-hadoop-on-ubuntu-22-04\/","url_meta":{"origin":1174,"position":2},"title":"How to Install Apache Hadoop on Ubuntu 22.04","author":"George B.","date":"September 4, 2023","format":false,"excerpt":"In today's digital age, the ability to process and analyze large data sets is crucial for businesses to stay competitive. Apache Hadoop, an open-source software framework, is a powerful tool that allows for distributed processing of large data sets across clusters of computers. This article will guide you through the\u2026","rel":"","context":"In &quot;Tutorials&quot;","block_context":{"text":"Tutorials","link":"https:\/\/www.virtono.com\/community\/category\/tutorial-how-to\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.virtono.com\/community\/wp-content\/uploads\/2023\/09\/How-to-Install-Apache-Hadoop-on-Ubuntu.png?fit=360%2C240&ssl=1&resize=350%2C200","width":350,"height":200},"classes":[]},{"id":3843,"url":"https:\/\/www.virtono.com\/community\/tutorial-how-to\/how-to-install-apache-spark-on-debian\/","url_meta":{"origin":1174,"position":3},"title":"How to Install Apache Spark on Debian","author":"George B.","date":"September 8, 2023","format":false,"excerpt":"In this article, we'll provide a step-by-step guide on how to install Apache Spark on Debian. Whether you're a newbie or an experienced user, this guide will make the installation process a breeze. If you're keen on big data processing, you've likely come across Apache Spark, a powerful open-source unified\u2026","rel":"","context":"In &quot;Tutorials&quot;","block_context":{"text":"Tutorials","link":"https:\/\/www.virtono.com\/community\/category\/tutorial-how-to\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.virtono.com\/community\/wp-content\/uploads\/2023\/09\/How-to-Install-Apache-Spark-on-Debian.png?fit=360%2C240&ssl=1&resize=350%2C200","width":350,"height":200},"classes":[]},{"id":235,"url":"https:\/\/www.virtono.com\/community\/knowledgebase\/lamp-technology\/","url_meta":{"origin":1174,"position":4},"title":"LAMP TECHNOLOGY","author":"Daniel Draga","date":"July 30, 2016","format":false,"excerpt":"LAMP STANDS FOR: L:Linux A:Apache M:MySQL P:PHP What is LAMP? LAMP is a shorthand term for a web application platform consisting of Linux, Apache, MySQL and one of Perl or PHP. Lamp is an\u00a0 OPENSOURCE Together, these open source tools provide a world-class platform for deploying web applications TECHNOLOGIES USED\u2026","rel":"","context":"In &quot;Knowledgebase&quot;","block_context":{"text":"Knowledgebase","link":"https:\/\/www.virtono.com\/community\/category\/knowledgebase\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.virtono.com\/community\/wp-content\/uploads\/2016\/07\/ic-1.png?fit=1200%2C750&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.virtono.com\/community\/wp-content\/uploads\/2016\/07\/ic-1.png?fit=1200%2C750&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/www.virtono.com\/community\/wp-content\/uploads\/2016\/07\/ic-1.png?fit=1200%2C750&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/www.virtono.com\/community\/wp-content\/uploads\/2016\/07\/ic-1.png?fit=1200%2C750&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/www.virtono.com\/community\/wp-content\/uploads\/2016\/07\/ic-1.png?fit=1200%2C750&ssl=1&resize=1050%2C600 3x"},"classes":[]},{"id":3472,"url":"https:\/\/www.virtono.com\/community\/tutorial-how-to\/how-to-install-apache-maven-on-centos-7\/","url_meta":{"origin":1174,"position":5},"title":"How to install Apache Maven on CentOS 7","author":"George B.","date":"June 8, 2023","format":false,"excerpt":"We will walk you through the process of installing Apache Maven on CentOS Linux. We will cover the installation of the necessary packages, configuring the environment variables, and demonstrate how to build projects using Maven. Apache Maven is a powerful build automation tool that plays a crucial role in managing\u2026","rel":"","context":"In &quot;Tutorials&quot;","block_context":{"text":"Tutorials","link":"https:\/\/www.virtono.com\/community\/category\/tutorial-how-to\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.virtono.com\/community\/wp-content\/uploads\/2023\/06\/How-to-install-Apache-Maven-on-CentOS-7.png?fit=600%2C330&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.virtono.com\/community\/wp-content\/uploads\/2023\/06\/How-to-install-Apache-Maven-on-CentOS-7.png?fit=600%2C330&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/www.virtono.com\/community\/wp-content\/uploads\/2023\/06\/How-to-install-Apache-Maven-on-CentOS-7.png?fit=600%2C330&ssl=1&resize=525%2C300 1.5x"},"classes":[]}],"_links":{"self":[{"href":"https:\/\/www.virtono.com\/community\/wp-json\/wp\/v2\/posts\/1174","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.virtono.com\/community\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.virtono.com\/community\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.virtono.com\/community\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.virtono.com\/community\/wp-json\/wp\/v2\/comments?post=1174"}],"version-history":[{"count":1,"href":"https:\/\/www.virtono.com\/community\/wp-json\/wp\/v2\/posts\/1174\/revisions"}],"predecessor-version":[{"id":1176,"href":"https:\/\/www.virtono.com\/community\/wp-json\/wp\/v2\/posts\/1174\/revisions\/1176"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.virtono.com\/community\/wp-json\/wp\/v2\/media\/1175"}],"wp:attachment":[{"href":"https:\/\/www.virtono.com\/community\/wp-json\/wp\/v2\/media?parent=1174"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.virtono.com\/community\/wp-json\/wp\/v2\/categories?post=1174"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.virtono.com\/community\/wp-json\/wp\/v2\/tags?post=1174"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}