No announcement yet.

New Solution for Sampling using TensorFlow and Phone Camera

  • Filter
  • Time
  • Show
Clear All
new posts

  • New Solution for Sampling using TensorFlow and Phone Camera

    The following document explains team 7641 M-SET BettaFish's solution for sampling using TensorFlowObjectDetection and the Android Phone Camera with Vuforia. We use similar methods to those in the external sample ConceptTensorFlowObjectDetection found in the FtcRobotController Package. We are able to achieve almost 100% accuracy.

    -Tej Bade

  • #2
    The link above doesn't work, please refer to the following link.


    • #3


      • #4
        Hi, I posted this basic approach several months ago, perhaps you might have missed it:

        We found several months ago that our uncertainty in the sampling was due to the silver mineral models failing due to the tape squares, so we switched over to simply only reading the gold mineral and using the relative x axis position within the frame to determine the location.

        We found that using the approach you have now won't give 100% as you have found, and is too slow. (viewing and filtering objects observed in the crater) We found it is far better to rigidly mount the phone so that it does not view any minerals in the crater at all, only the samples at the top of the frame. Once we did that, we were at 100% in hundreds of trials and at all competitions.

        (Except we had one failure during competition unrelated to our code. One of our team members forgot to make sure the phone was alive and displaying the image on the screen, if it goes to sleep Vuforia goes dead and TensorFlow cannot work)


        • #5
          Like 11343_Mentor we had the camera mounted high at the back of the robot, in landscape mode, so that it could 'see' the three minerals, and the camera was tilted down so that it did not see anything in the crater.

          Originally we used the tensorflow concept sample code, but when we mounted the final collector design it obscured the left most mineral so we modified our program to use the x position of the mineral to decide which of the three positions it was in, and then for all recognized minerals (which could be greater than three) we added 1 to the position count if a gold was detected and subtracted 1 if a silver was detected.

          We started the TensorFlow detector at start() paused 0.5 seconds to give the detector a chance to settle, then 'landed' for a couple of seconds, still running the detector. About half way through the landing process the camera would start to 'see' the left mineral position, but vibrations during the landing process slowed down the detection of new objects.

          On landing the robot would change angle slightly causing the apparent object positions to shift vertically which could result in two detections in a position bin as both the old and new mineral position were reported.

          Then we looked at the counts in each of the three position bins and used the most positive (least negative) as the gold position.

          This worked in every game in the final competition where we had this implementation, but i am reluctant to say 100% with a sample size of 8 matches.