Missing moveit_task_constructor_msgs/SampleGraspPosesAction.h

[marcpujol-leitat]

I am trying to follow the moveit tutorials to install Moveit Deep Grasps but compiler fails saying this header is missing.

Is this intended and I need to replace them?

Thank you!

[rhaschke]

Unfortunately, we didn't yet merge #196, which is the basis for the mentioned tutorial. Also, in https://github.com/ros-planning/moveit_task_constructor/pull/196#discussion_r592675803 it was suggested to use an existing action instead of a new one. This essentially replaced SampleGraspPosesAction with grasping_msgs::GraspPlanningAction. So here is what I suggest:

1. Merge #196 into your MTC repo
2. Update the demo to use grasping_msgs::GraspPlanningAction
3. Fix all remaining issues.
4. File a PR against https://github.com/PickNikRobotics/deep_grasp_demo and/or https://ros-planning.github.io/moveit_tutorials to incorporate all required changes into the demo/tutorial.

[qyp-robot]

Hello, when I was reproducing the GPD algorithm, I followed the Deep Grasp Demo operation, but I encountered a missing header file error when compiling, how can I solve it? Hope you can help me! Thank you very much! Looking forward to your reply, thank you! Please be more detailed, thank you

[rhaschke]

@qyp-robot: Didn't my suggestion point you in the right direction? If not, please provide a detailed error description (error messages, your environment, etc.)

[qyp-robot]

Hello, do I modify it like this here, just the following two links： 1.https://github.com/ros-planning/moveit_task_constructor/pull/196/commits/a2988995bcb85a099ca6766c709b90eac40d4432 2.https://github.com/ros-planning/moveit_task_constructor/pull/196/commits/19fb72c81796303336d399905576a7de69602fc0

[qyp-robot]

Hello, when I make changes according to this, it can be compiled normally; But after I ran roslaunch moveit_task_constructor_gpd gpd_demo.launch, moveit did not capture, and the following error was reported on the terminal:

[ INFO] [1626148048.631361699]: Generate grasp goal now active runtime(computeNormals): 0.3677 camera: 0, #indices: 3998, #normals: 3998 Calculated 3998 surface normals in 0.3699s (mode: OpenMP). Reversing direction of normals that do not point to at least one camera ... reversed 0 normals runtime (reverse normals): 4.5744e-05 Estimating local reference frames ... Estimated 50 frames in 0.0006s. Finding hand poses ... Found 50 hand sets in 0.32s ====> HAND SEARCH TIME: 0.321938 Generated 50 hand sets. Filtering grasps outside of workspace ... Number of grasp candidates within workspace and gripper width: 400 Number of grasp candidates with correct approach direction: 400 neighborhoods search time: 0.0382 Created 400 images in 1.1584s [grasp_cloud_detection-2] process has died [pid 410550, exit code -11, cmd /home/qyp/ws_grasp/devel/lib/moveit_task_constructor_gpd/grasp_cloud_detection __name:=grasp_cloud_detection __log:=/home/qyp/.ros/log/feb1051a-e38c-11eb-a2be-5378e1583e88/grasp_cloud_detection-2.log]. log file: /home/qyp/.ros/log/feb1051a-e38c-11eb-a2be-5378e1583e88/grasp_cloud_detection-2*.log

[rhaschke]

@bostoncleek, can you help on this issue?

[qyp-robot]

@bostoncleek，你能帮助解决这个问题吗？ First of all, thank you very much for your reply, I still got the following error when I tested: [grasp_cloud_detection-2] process has died [pid 411886, exit code -11, cmd /home/qyp/ws_grasp/devel/lib/moveit_task_constructor_gpd/grasp_cloud_detection __name:=grasp_cloud_detection __log:=/home/qyp/.ros/log/feb1051a -e38c-11eb-a2be-5378e1583e88/grasp_cloud_detection-2.log]. log file: /home/qyp/.ros/log/feb1051a-e38c-11eb-a2be-5378e1583e88/grasp_cloud_detection-2*.log

[marcpujol-leitat]

Thank you @rhaschke for the fast response!

I have already merged the PR into my repo and now I am trying to modify the deep_grasp_demo package so it works with the changes.

I am having some troubles with that:

• Inside moveit_task_constructor_(gpd and dexnet) there are header files named grasp_detection.h which create a SimpleActionServer. I understand I need to modify those so they use the GraspPlanningAction. However, GraspPlanningFeedback and Result are missing members that the .cpp is using.
• In the commits of the PR I can see @bostoncleek has also created a class named GraspProvider. Should I be using that inside the deep_pick_and_place.cpp demo, for example in the Generate Grasp Pose stage.

Thanks again!

[rhaschke]

I'm afraid I need to forward your questions to @bostoncleek. He implemented all this code originally.

[qyp-robot]

Thank you very much for your reply. I am also working on it. If you have the latest progress, please let me know in time. Thank you very much!

------------------ 原始邮件 ------------------ 发件人: "ros-planning/moveit_task_constructor" @.>; 发送时间: 2021年7月13日(星期二) 下午5:53 @.>; @.@.>; 主题: Re: [ros-planning/moveit_task_constructor] Missing moveit_task_constructor_msgs/SampleGraspPosesAction.h (#281)

I'm afraid I need to forward your questions to @bostoncleek. He implemented all this code originally.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[bostoncleek]

You should use GraspProvider in both moveit_task_constructor_(gpd and dexnet) packages.

You will also need to update the tutorial and demo to use grasping_msgs::GraspPlanningAction. If GraspPlanningFeedback and Result are missing make sure you include the grasping_msgs::GraspPlanningAction action message in you code. Check out the action lib tutorials for help.

If you have more questions please let me know. I am trying to get permission from my employer to update the demo and tutorial. But if you finish the modifications rhaschke made above please submit a PR and I will review it.

@marcpujol-leitat

[marcpujol-leitat]

Hi,

I have created the PR so you can see the changes I've made.

Thank you!!

[qyp-robot]

Hi, I have changed the New grasp pose generator stage as per here: https://github.com/ros-planning/moveit_task_constructor/pull/196/commits/a2988995bcb85a099ca6766c709b90eac40d4432 This problem has been solved: header is missing

Second, I make changes according to here, Rename stage to Grasp Provider: https://github.com/ros-planning/moveit_task_constructor/pull/196/commits/19fb72c81796303336d399905576a7de69602fc0 But there is a problem when compiling, what should I do, I look forward to your help!

@marcpujol-leitat @bostoncleek.

[qyp-robot]

Hello @bostoncleek,@marcpujol-leitat,@rhaschke When I change according to here, Rename stage to Grasp Provider: https://github.com/ros-planning/moveit_task_constructor/commit/19fb72c81796303336d399905576a7de69602fc0；

Do I also need to update the tutorial and demo to use grasping_msgs::GraspPlanningAction? Then compile it? Thank you all!

[marcpujol-leitat]

Hi @qyp-robot,

Now that you have solved the header problems, I think you have to modify the deep_grasp_demo so it works with the changes made in moveit_task_constructor according to the links you have sent.

[qyp-robot]

Thank you for your prompt reply!@marcpujol-leitat I have followed your prompt: https://github.com/PickNikRobotics/deep_grasp_demo/pull/11/commits/971d3db105473eb93ab948ff9cf675905df3f951

The demo to use grasping_msgs::GraspPlanningAction has been updated, but when I was compiling, an error occurred, such as:

rror: ‘const struct grasping_msgs::GraspPlanningFeedback_<std::allocator >’ has no member named ‘grasp_candidates’ 175 | if (feedback->grasp_candidates.size() != feedback->costs.size()) { | ~~~~~~~~~~^~~~~~~~~~~~~~~~

error: ‘const struct grasping_msgs::GraspPlanningFeedback_<std::allocator >’ has no member named ‘costs’ 175 | if (feedback->grasp_candidates.size() != feedback->costs.size()) { | ~~~~~~~~~~^~~~~

I checked carefully. In my /opt/ros/noetic/share/grasping_msgs/action, there is a file GraspPlanning.action. I don't know how to solve this compilation error. I hope you can help me! Thank you very much!

[marcpujol-leitat]

During the changes you made in the moveit_task_constructor, you substituted the original action with GraspPlanningAction which no longer has "grasp_candidates" inside its feedback. You can find there message in the documentation: Grasp Planning Action As bostoncleek already suggested, if you need help with that, you should check the Action Lib Tutorials

[qyp-robot]

Thank you very much!@marcpujol-leitat As you said, this is the problem, I am revising it, thank you! If I have any questions, I will ask you again. Thank you very much for your reply!

[qyp-robot]

Hello, @marcpujol-leitat when I changed the grass_provider, he originally used the information in SampleGraspPoses.action, the information is as follows:

goal sent to client

string action_name

result sent to server

string grasp_state

feedback sent to server

grasp poses

geometry_msgs/PoseStamped[] grasp_candidates

cost of each grasp

float64[] costs

But the information in GraspPlanning.action is this:

This action is used when planning grasps for a single,

already known object, one object at a time.

Object for which grasp planning is requested

Object object

Name of group to plan with (optional)

string group_name

All grasps

moveit_msgs/Grasp[] grasps

Grasps found thus far

moveit_msgs/Grasp[] grasps

In grape_provider.h, I am not quite clear about what information in GraspPlanning.action should be used to replace the original costs message, and what should be its action_name; I’m not sure how to change it, can you help me? Thank you!

[marcpujol-leitat]

Hi @qyp-robot,

If you go to my Pull Request you will see the changes I made to grasp_detection.cpp. There you can see what I did to solve what you are mentioning.

However, it is still not working and I am waiting for bostoncleek to review the changes, so do not expect it to work propery.

[qyp-robot]

Thank you very much for your reply.@marcpujol-leitat It was at night on our side before, and I didn’t have time to reply you in time. I'm sorry, I will go for the test, thank you very much!

[qyp-robot]

Hello! @bostoncleek.
Can you take a look at the changes of pr from @marcpujol-leitat： https://github.com/PickNikRobotics/deep_grasp_demo/pull/11/files Thank you very much!

[qyp-robot]

Hi,@marcpujol-leitat Have you solved the problem? The following message appears when I run the terminal, and it is stuck here:

[INFO] [1626318339.260666630]: Loading task parameters [INFO] [1626318339.270120710]: Initializing task pipeline [INFO] [1626318339.278097396]: Loading robot model'panda'...

I am also very anxious. I have checked almost all the information but have not resolved it. Do you have any directions? Thank you!

[marcpujol-leitat]

Hi @qyp-robot,

This happens when you run moveit_task_constructor_demo demo.launch?

It is working for me. Did you follow the tutorials?

I am using Ubuntu 20.04 and ROS Noetic. In the same workspace I have the moveit_task_constructor I also have the moveit repository (noetic-devel) and panda_moveit_config.

Hope it helps.

[qyp-robot]

Thank you for your reply!@marcpujol-leitat The first thing I run is: roslaunch moveit_task_constructor_demo demo.launch Then I ran the GPD package: roslaunch moveit_task_constructor_gpd gpd_demo.launch

Then the terminal stays here all the time: ============= CLASSIFIER ====================== model_file: weights_file: /home/qyp/shenduxuexi/gpd/models/lenet/15channels/params batch_size: 1

============ CANDIDATE FILTERING ============= candidate_workspace: -1.00 1.00 -1.00 1.00 -1.00 1.00 min_aperture: 0.0000 max_aperture: 0.8500

============ CLUSTERING ===================== min_inliers: 1

[INFO] [1626346266.686664813]: Loading task parameters [INFO] [1626346266.700117082]: Initializing task pipeline [INFO] [1626346266.703910978]: Loading robot model'panda'...

Moveit did not perform crawling, etc. I think there should be no problem with this gpd part, is there still a problem with the moveit_task_constructor package?

[qyp-robot]

Hi@marcpujol-leitat After I followed your changes before, it is here to change: https://github.com/PickNikRobotics/deep_grasp_demo/pull/11/files

Later, when compiling, the terminal reported an error and could not compile. Later, I made changes according to this: https://github.com/ros-planning/moveit_task_constructor/pull/196/commits/ae376b6fc703b75bcf41e6d6a46bba0c75b21311

Compilation is successful, and then there is a problem with the above operation

[marcpujol-leitat]

In my case it runs past where you are stuck and gets stuck somewhere else.

I'm sorry but I don't know how to help you, the only thing I can tell you is to get bostoncleek's fork of moveit_task_constructor instead of the commit you are mentioning:

• [1] git clone https://github.com/ros-planning/moveit_task_constructor.git
• [2] git remote add pr-mtc https://github.com/bostoncleek/moveit_task_constructor
• [3] git fetch pr-mtc
• [4] git merge pr-mtc/pr-deep_grasp_stage

This way you will have the changes he made.

[qyp-robot]

Hello,@marcpujol-leitat Thanks for the tip, I checked this branch: https://github.com/ros-planning/moveit_task_constructor/compare/master...bostoncleek:pr-deep_grasp_stage He merged all the previous changes, there are a total of 8 file changes, right, all the changes made are here, right? I think it is the same as what you suggested! The same is true for my previous modification!

I branched his pr-deep_grasp_stage code: https://github.com/bostoncleek/moveit_task_constructor/tree/pr-deep_grasp_stage

Copy it to my workspace. At the moment, my workspace has the following 2 function packages: moveit_task_constructor panda_moveit_config

Compiled and can run the following two launches normally: roslaunch moveit_task_constructor_demo demo.launch roslaunch moveit_task_constructor_demo pickplace.launch

So I think that the moveit_task_constructor function package and the basic repair are completed, and the problem may still be in the deep_grasp_demo function package. I will follow your prompts again to modify the code in the deep_grasp_demo function package.

Is my understanding right? Thank you for your continued help. Thank you very much!

[marcpujol-leitat]

Yes, I think we should wait for bostoncleek to review our changes.

Well maybe @rhaschke could help with my issue, as it is a problem regarding the plan function of the class moveit::task_constructor::task

After running the moveit_task_constructor_demo demo.launch and the moveit_task_constructor_gpd gpd_demo.launch, it looks like it is able to create the grasps and select the best ones but gets stuck there, not executing the task. The command window looks like this:

I have checked the code and it gets stuck here, in the task_.plan(max_solutions)

Where task_ is an object of the class moveit::task_constructor::task.

Do you have any idea of why this may be happening?

Thank you!

[qyp-robot]

Sorry, I don't know how to solve it at the moment, I will try it, and if there is any progress, I must contact you in time. Thank you