Peer Session Badge

My assignment: Hyperparameter search

Timespace of ML model

  • Entropy
    1. initial state (low entropy)
    2. ML process (high entropy: many possible states)
    3. terminal state (low entropy)
  • Parameter, Hyperparameter search
    • Grid search: Stochastic
      • SHA (Successive Having Algorithm)
    • Random search(2012)): Non-stochastic
    • Surrogate model
      • Gaussian process
  • Neural Architecture Search (NAS)
    • automatic NAS with reinforcement learning (ICLR 2017)
  • NAS for edge devices
    • MnasNet
    • Proxylessnas
    • Once-for-all
    • MobileNets