• DocumentCode
    3059324
  • Title

    On the memory required to compute functions of streaming data

  • Author

    Viswanathan, Krishnamurthy

  • Author_Institution
    HP Labs., Palo Alto, CA, USA
  • fYear
    2010
  • fDate
    13-18 June 2010
  • Firstpage
    196
  • Lastpage
    200
  • Abstract
    We consider the problem of computing functions of data streams while employing limited memory in a standard information-theoretic framework. A streaming system with memory constraint has to observe a collection of sources X1, X2,...,Xm sequentially, store synopses of the sources in memory, and compute a function of the sources based on the synopses. We establish a correspondence between this problem and a functional source coding problem in cascade/line networks. For the general functional source coding problem in cascade networks, we derive inner and outer bounds, and for distributions satisfying certain properties, we characterize the achievable rate-region exactly for the computation of any function. As a result of the correspondence we established, this result also characterizes the minimum amount of memory required to compute the function in a streaming system. We briefly discuss the implications of this result for the problem of distinct value computation.
  • Keywords
    information theory; media streaming; source coding; storage management; cascade network; computing function; data streaming system; distinct value computation; functional source coding problem; line network; memory constraint; standard information theoretic framework; Computer network management; Computer networks; Computer science; Data processing; Distributed computing; Engines; Memory management; Monitoring; Source coding; Telecommunication computing;
  • fLanguage
    English
  • Publisher
    ieee
  • Conference_Titel
    Information Theory Proceedings (ISIT), 2010 IEEE International Symposium on
  • Conference_Location
    Austin, TX
  • Print_ISBN
    978-1-4244-7890-3
  • Electronic_ISBN
    978-1-4244-7891-0
  • Type

    conf

  • DOI
    10.1109/ISIT.2010.5513246
  • Filename
    5513246